Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The copyFileToPodAsync method causes the process to fail to close! #3401

Open
swzaaaaaaa opened this issue May 14, 2024 · 4 comments
Open

Comments

@swzaaaaaaa
Copy link

swzaaaaaaa commented May 14, 2024

Describe the bug
Hello. I refer to the link "https://github.com/kubernetes-client/java/blob/master/util/src/main/java/io/kubernetes/client/Copy.java" to write file upload Code, found that the code will get stuck in "proc.waitFor(); "This step. I think it is not the output stream that can not be closed all the time. Is there any good method?
I tried to comment out "proc.waitFor()" and "proc.destroy()" in full, and I was able to complete the file upload, but this should cause problems such as memory leaks.

Kubernetes Version
e.g. 1.15.0

Java Version
e.g. Java 8

To Reproduce


public void copyFileToPod(
      String namespace, String pod, String container, byte[] src, Path destPath)
      throws ApiException, IOException {
    try {
      int exit = copyFileToPodAsync(namespace, pod, container, src, destPath).get();
      if (exit != 0) {
        throw new IOException("Copy failed: " + exit);
      }
    } catch (InterruptedException | ExecutionException ex) {
      throw new IOException(ex);
    }
  }

  public Future<Integer> copyFileToPodAsync(
      String namespace, String pod, String container, byte[] src, Path destPath)
      throws ApiException, IOException {
    // Run decoding and extracting processes
    final Process proc = execCopyToPod(namespace, pod, container, destPath);

    try (ArchiveOutputStream archiveOutputStream =
        new TarArchiveOutputStream(proc.getOutputStream())) {

      ArchiveEntry tarEntry = new TarArchiveEntry(new File(destPath.getFileName().toString()));
      ((TarArchiveEntry) tarEntry).setSize(src.length);

      archiveOutputStream.putArchiveEntry(tarEntry);
      Streams.copy(new ByteArrayInputStream(src), archiveOutputStream);
      archiveOutputStream.closeArchiveEntry();

      return new ProcessFuture(proc);
    }
  }

  private Process execCopyToPod(String namespace, String pod, String container, Path destPath)
      throws ApiException, IOException {
    // TODO: This assumes Linux and won't work on Windows Containers (for many reasons...)
    String parentPath = destPath.getParent() != null ? destPath.getParent().toString() : ".";
    parentPath = parentPath.replace("\\", "/");
    return this.exec(
        namespace,
        pod,
        new String[] {"sh", "-c", "tar -xmf - -C " + parentPath},
        container,
        true,
        false);
  }

  private static class ProcessFuture implements Future<Integer> {
    private Process proc;

    ProcessFuture(Process proc) {
      this.proc = proc;
    }

    // TODO: support cancelling?
    public boolean cancel(boolean interupt) {
      return false;
    }

    public boolean isCancelled() {
      return false;
    }

    public Integer get() throws InterruptedException {
      **proc.waitFor(); /////here**
      proc.destroy();
      return proc.exitValue();
    }

    public Integer get(long timeout, TimeUnit unit) throws InterruptedException {
      proc.waitFor(timeout, unit);
      proc.destroy();
      return proc.exitValue();
    }

    public boolean isDone() {
      return proc.isAlive();
    }
  }

Expected behavior
A clear and concise description of what you expected to happen.

KubeConfig
If applicable, add a KubeConfig file with secrets redacted.

@lm-lan
Copy link

lm-lan commented May 16, 2024

You can manually close it after the upload is completed

Future<Integer> future = copy.copyFileToPodAsync(namespace, podInfo.getPodName(), null, Paths.get(srcPath), Paths.get(descPath));
  future.get(1, TimeUnit.NANOSECONDS); 

@swzaaaaaaa
Copy link
Author

You can manually close it after the upload is completed

Future<Integer> future = copy.copyFileToPodAsync(namespace, podInfo.getPodName(), null, Paths.get(srcPath), Paths.get(descPath));
  future.get(1, TimeUnit.NANOSECONDS); 

Hello, I use this method, but it will report an Error every time: "Error on flush, java.io.IOException: Socket is closed!" Is this normal? Where the file is finished uploading.

@lm-lan
Copy link

lm-lan commented May 21, 2024

You can manually close it after the upload is completed

Future<Integer> future = copy.copyFileToPodAsync(namespace, podInfo.getPodName(), null, Paths.get(srcPath), Paths.get(descPath));
  future.get(1, TimeUnit.NANOSECONDS); 

Hello, I use this method, but it will report an Error every time: "Error on flush, java.io.IOException: Socket is closed!" Is this normal? Where the file is finished uploading.

Streams.copy(new ByteArrayInputStream(src), archiveOutputStream);
This line of code executes the upload,
Do you have a complete stack, or try uploading a small file

@swzaaaaaaa
Copy link
Author

The same is true when I uploaded a small file (less than 1M). I suspect that pod could not know the end of the uploaded file, so it would hang on forever, which would cause this problem when we proactively closed.
As the question says “https://github.com/kubernetes-client/java/issues/1822”

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants