Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing to upload smaller streams having .WithObjectSize() as max possible stream size #1068

Open
w7rus opened this issue May 1, 2024 · 0 comments

Comments

@w7rus
Copy link

w7rus commented May 1, 2024

I'm having an ASP.NET Controller incoming form data and request size is limited with following attributes:

//Assume Consts.AvatarMaxFileSize is 10MB

[RequestSizeLimit(Consts.AvatarMaxFileSize + 4096)]
[RequestFormLimits(MultipartBodyLengthLimit = Consts.AvatarMaxFileSize)]
...

Then the minioClient instance forces me to use .WithObjectSize(), assuming limits set on the controller, stream size would be less or equal to Consts.AvatarMaxFileSize, so:

...

//Upload file
var putObjectArgs = new PutObjectArgs()
    .WithBucket(bucketName)
    .WithStreamData(file)
    .WithObject(fileName)
    .WithContentType(fileContentType);
    .WithObjectSize(Consts.AvatarMaxFileSize);

await _minioClient.PutObjectAsync(putObjectArgs, cancellationToken).ConfigureAwait(false);

Which throws an exception when validating CompleteMultipartUploadArgs since ETags dictionary is null, because of following block of code at Minio.MinioClient.PutObjectPartAsync

// This shouldn't happen where stream size is known.
if (partCount != numPartsUploaded && args.ObjectSize != -1)
{
    var removeUploadArgs = new RemoveUploadArgs()
        .WithBucket(args.BucketName)
        .WithObject(args.ObjectName)
        .WithUploadId(args.UploadId);
    await RemoveUploadAsync(removeUploadArgs, cancellationToken).ConfigureAwait(false);
    return null;
}

//args.ObjectSize = 52428800 - stream max possible size
//partSize = 16777216 - stream actual size
//partCount = 4
//numPartsUploaded = 1

This happens when stream size is smaller than maximum possible, causing partCount != numPartsUploaded to return null ETags, removing an object

I can not determine actual file size since i read multipart request as stream, dropping it in memory/temp file does not sound like a good solution, rather than streaming directly

Can this check be removed since it is excessive?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant