Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minio 6.0.2: Unexpected Behavior with Unavailable Endpoints #1041

Closed
teun2408 opened this issue Mar 26, 2024 · 6 comments · Fixed by #1057
Closed

Minio 6.0.2: Unexpected Behavior with Unavailable Endpoints #1041

teun2408 opened this issue Mar 26, 2024 · 6 comments · Fixed by #1057

Comments

@teun2408
Copy link

Issue Summary:

Upgrading from Minio .NET SDK version 6.0.1 to 6.0.2 has introduced unexpected behavior when interacting with non-existent Minio server endpoints. Specifically, methods like BucketExistsAsync and GetObjectAsync do not behave as expected when provided with endpoints that are not running Minio servers. Instead of throwing exceptions or indicating errors, they return unexpected results, such as always returning true for bucket existence checks and returning empty streams for object downloads.

Steps to Reproduce:

  1. Use Minio .NET SDK version 6.0.2.
  2. Configure the SDK with an endpoint that does not host a Minio server (e.g., "127.0.0.1:11111").
  3. Use the BucketExistsAsync method to check for the existence of a bucket that doesn't exist.
  4. Use the GetObjectAsync method to download an object from a non-existent bucket.
using Minio;

var s3Client = new MinioClient()
    .WithEndpoint("127.0.0.1:11111")
    .WithCredentials("accesskey", "secretkey")
    .Build();

//Should throw an exception, but it does not in 6.0.2, in 6.0.1 it throws a System.AggregateException
//Instead, in 6.0.2 it just returns true even though the Minio server does not exist, nor the bucket.
bool bucketExists = s3Client.BucketExistsAsync(new Minio.DataModel.Args.BucketExistsArgs().WithBucket("bucketname")).Result;

if (bucketExists)
{
    Console.WriteLine("Bucket exists according to Minio");
}

MemoryStream stream = new MemoryStream();

// An exception should be thrown here, but it is not, instead, it returns an empty stream even though neither the server, bucket, or object exists.
await s3Client.GetObjectAsync(new Minio.DataModel.Args.GetObjectArgs().WithBucket("bucketname").WithObject("objectname").WithCallbackStream(x => x.CopyTo(stream)));

Console.WriteLine($"Stream length: {stream.Length}");

Expected Behavior:

  • The BucketExistsAsync method should throw an exception or return false when provided with an endpoint that does not host a Minio server.
  • The GetObjectAsync method should throw an exception when attempting to download an object from a non-existent bucket.

Actual Behavior:

  • When using BucketExistsAsync with a non-existent Minio server endpoint, it consistently returns true, incorrectly indicating the existence of the bucket.
  • When using GetObjectAsync with a non-existent bucket or object, it returns an empty stream instead of throwing an exception or indicating an error.

Environment Details:

  • Minio .NET SDK version: 6.0.2
  • Operating System: Windows 11
  • .NET 8
@cvetomir-todorov
Copy link

cvetomir-todorov commented Mar 31, 2024

Happens to me as well. Works with 6.0.1, breaks with 6.0.2. I am using Ubuntu 22.04 and .NET 8.

@OK111
Copy link

OK111 commented Apr 3, 2024

Also PutObjectAsync doesn't throw any errors in 6.0.2 .NET 8

@guillaumeagile
Copy link

The same. I tried to check the behavior of the driver with test containers, and got the same issue. I finally realized that the problem doesn't come from TestContainers, but from the driver.

@guillaumeagile
Copy link

guillaumeagile commented Apr 7, 2024

Observed exactly that weird behavior when the endpoint is unreachable
like if the driver was talking to a "ghost" server that says yes to everything and that acts like a black hole when you upload to it.

Using .Net 8, TestContainers , xUnit and 6.0.2
service.AddMinio(configureClient => configureClient .WithEndpoint(uri) .WithCredentials(MinIoAdapter.AccessKey, MinIoAdapter.SecretKey) .WithSSL(false) .Build() );
where uri is not pointing to a running Minio instance.

await _minioClient.BucketExistsAsync(beArgs).ConfigureAwait(false);
or await _minioClient.PresignedGetObjectAsync(presignedGetObjectArgs).ConfigureAwait(false)
always returns something (with an empty container)

_minioClient.GetObjectAsync(getObjectArgs).ConfigureAwait(true)
always returns a 0 byte length memoryStream or file, even if I has uploaded an object before (with the right name)

I'm initializing my container with

_minioContainer = new MinioBuilder() .WithUsername(MinIoAdapter.AccessKey) .WithPassword(MinIoAdapter.SecretKey) .WithPortBinding(9000, true) .WithCleanUp(true) .Build();
it's also obvious when I remove the .WithUsername(MinIoAdapter.AccessKey) .WithPassword

holterbades added a commit to holterbades/minio-dotnet that referenced this issue Apr 18, 2024
Response error handling was unintentionally removed in a previous commit. Added it back to the `ExecuteTaskCoreAsync` method.
Fixed `NullReferenceException`-bug in the `HandleIfErrorResponse` method - `throw response.Exception;` does not make sense if the exception is null.
holterbades added a commit to holterbades/minio-dotnet that referenced this issue Apr 18, 2024
Response error handling was unintentionally removed in a previous commit. Added it back to the `ExecuteTaskCoreAsync` method.
Fixed `NullReferenceException`-bug in the `HandleIfErrorResponse` method - `throw response.Exception;` does not make sense if the exception is null.
@holterbades
Copy link
Contributor

Imho this is a critical problem and this Issue should get the name "No error handling" because there is no actual error handling active anymore.

I encountered the following issues:

When i download a file with valid bucketName and objectName but the user/access-token does not have permission to download the object (e.g. due to a policy) or a wrong region was set in the client, no exception is thrown. So i can't verify that the download failed. The funny thing is, that instead of the object-content on the storage, the xml error response gets written to the output file.

var getObject = new GetObjectArgs()
    .WithBucket(bucketName)
    .WithObject(objectName)
    .WithFile("C:\\temp\\test2.txt");
var stat = await minio.GetObjectAsync(getObject);

Similar to this example, if i stream the object content directly to another destination (e.g. another S3 object store), i also get the error message content instead of an exception streamed to the destination. Furthermore, if i encounter an exception in the streaming action like outlined below, this exception is also catched and not delegated to the user code, which also makes it impossible to handle errors.

var getObject = new GetObjectArgs()
    .WithBucket(bucketName)
    .WithObject(objectName)
    .WithCallbackStream(stream =>
    {
        throw new Exception("Error uploading file to storage.");
    });
var stat = await minio.GetObjectAsync(getObject);

Imagine you use this action to stream the blob directly to an deep archive with retention for 7 years - in this case you get files with error messages instead of your intended object content - which you can't delete because of retention policies set on the target.

The only way you could verify that an error occoured is by validating the ObjectStat.ExtraHeaders returned by GetObjectAsync(...). But this clearly can't be a solution because it is provider dependent and would be a breaking change for clients out there.

I traced back the problem to changes in the commit 9772a373b92d81808118edbdbd653d301f4502ac, but I don't know what the intention for those changes was because it's not well documented neither in the commit nor in the linked issue (#987). But the problem is, that in this commit, the method call to HandleIfErrorResponse was unintentionally removed (this method has no other calls and if this removal was intended, the method HandleIfErrorResponse should have also been removed).

I opened a PullRequest #1057 to fix this issue.

@holterbades
Copy link
Contributor

I think, the other problems mentioned in this issue should also get resolved with my fix, because this method ExecuteTaskAsync(...) -> ExecuteTaskCoreAsync(...) is the core method used by all operations.

holterbades added a commit to holterbades/minio-dotnet that referenced this issue Apr 22, 2024
Response error handling was unintentionally removed in a previous commit. Added it back to the `ExecuteTaskCoreAsync` method.
Fixed `NullReferenceException`-bug in the `HandleIfErrorResponse` method - `throw response.Exception;` does not make sense if the exception is null.
ebozduman pushed a commit that referenced this issue May 24, 2024
Response error handling was unintentionally removed in a previous commit. Added it back to the `ExecuteTaskCoreAsync` method.
Fixed `NullReferenceException`-bug in the `HandleIfErrorResponse` method - `throw response.Exception;` does not make sense if the exception is null.
ebozduman pushed a commit to ebozduman/minio-dotnet that referenced this issue May 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants