Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fetcher archive code why use DefaultZip #2936

Open
bigbird-0101 opened this issue May 10, 2024 · 10 comments
Open

fetcher archive code why use DefaultZip #2936

bigbird-0101 opened this issue May 10, 2024 · 10 comments

Comments

@bigbird-0101
Copy link

Is your feature request related to a problem? Please describe.
problem:
one env can only build and package one program
error message: error archiving zip file: creating zip: zip archive is already created for writing"

Additional context
Why does the fetcher archive code use the default DefaultZip instead of creating a new one, so that one env can only build and package one program?
Why not?

// archive zips the contents of directory at src into a new zip file
// at dst (note that the contents are zipped, not the directory itself).
func (fetcher *Fetcher) archive(src string, dst string) error {
	var files []string
	target, err := os.Stat(src)
	if err != nil {
		return errors.Wrap(err, "failed to zip file")
	}
	if target.IsDir() {
		// list all
		fs, _ := os.ReadDir(src)
		for _, f := range fs {
			files = append(files, filepath.Join(src, f.Name()))
		}
	} else {
		files = append(files, src)
	}
	zipFile := archiver.NewZip()
	defer zipFile.Close()
	return zipFile.Archive(files, dst)
}

// unarchive is a function that unzips a zip file to destination
func (fetcher *Fetcher) unarchive(src string, dst string) error {
	zipFile := archiver.NewZip()
	defer zipFile.Close()
	err := zipFile.Unarchive(src, dst)
	if err != nil {
		return fmt.Errorf("failed to unzip file: %w", err)
	}
	return nil
}
@bigbird-0101
Copy link
Author

bigbird-0101 commented May 11, 2024

my update code already build images, name is bigbird0101/fission-fetcher:v1.17.3
and This images is already running online.
To those who are destined

@soharab-ic
Copy link
Contributor

@bigbird-0101 I am able to build and package multiple programs with one env.

$ fission env list
NAME IMAGE               BUILDER_IMAGE           POOLSIZE MINCPU MAXCPU MINMEMORY MAXMEMORY EXTNET GRACETIME NAMESPACE
go   fission/go-env-1.16 fission/go-builder-1.16 3        0      0      0         0         false  0         default

$ fission pkg list
NAME                                          BUILD_STATUS ENV LASTUPDATEDAT       NAMESPACE
zip-test-9f78b747-8c6c-43fc-bc99-4ea8f87a4439 succeeded    go  03 Jun 24 12:18 IST default
zip-test-7ce3a8ae-f533-4100-8cce-e2f94bcaeb03 succeeded    go  03 Jun 24 12:15 IST default
zip-go-40c5635d-3331-4b86-a7cd-289cb44aa601   succeeded    go  03 Jun 24 12:14 IST default
hello-go-e207c8e1-49de-4dd6-9aa7-90b8b70cd2e2 succeeded    go  03 Jun 24 11:59 IST default

$ fission fn list
NAME     ENV EXECUTORTYPE MINSCALE MAXSCALE MINCPU MAXCPU MINMEMORY MAXMEMORY SECRETS CONFIGMAPS NAMESPACE
hello-go go  poolmgr      0        0        0      0      0         0                            default
zip-go   go  poolmgr      0        0        0      0      0         0                            default
zip-test go  poolmgr      0        0        0      0      0         0                            default

Please provide the steps to reproduce the issue.

@bigbird-0101
Copy link
Author

@soharab-ic I know you means, but You can try sending multiple packaging requests in one second

@soharab-ic
Copy link
Contributor

@bigbird-0101 I saw this issue after sending 14 request in one second.

Error uploading deployment package: Internal error - error archiving zip file: file already exists: /packages/zip-test12-c39404ab-6fbc-40a2-8592-ebd9ea9ea6a8-ixnecf-lvvg6k.zip

Please raise PR.

@soharab-ic
Copy link
Contributor

I tried above changes you have mentioned but it did not work for me.
Even DefaultZip calls NewZip() for a zip instance.

var DefaultZip = NewZip()

https://github.com/mholt/archiver/blob/v3.5.1/zip.go#L711

@bigbird-0101
Copy link
Author

bigbird-0101 commented Jun 3, 2024

@soharab-ic I only changed these four lines, I will try pr, you can also change fetcher image (use bigbird0101/fission-fetcher:v1.17.3)
image

@soharab-ic
Copy link
Contributor

@bigbird-0101 I tried your fetcher image as well as re-tried your patch, it's working.
Maybe something was wrong with my installation yesterday i.e., why the patch did not work.

Please raise PR.

@soharab-ic
Copy link
Contributor

@bigbird-0101 Should i raise the PR?

@bigbird-0101
Copy link
Author

@soharab-ic sorry, I try PR,but this #2955 is blocked

@soharab-ic
Copy link
Contributor

@bigbird-0101 Thanks for raising the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants