Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feat]: Add Supabase as S3 provider for Coolify #2114

Open
ukashazia opened this issue May 2, 2024 · 17 comments
Open

[Feat]: Add Supabase as S3 provider for Coolify #2114

ukashazia opened this issue May 2, 2024 · 17 comments
Assignees

Comments

@ukashazia
Copy link

ukashazia commented May 2, 2024

Objectives:

  • create a s3 bucket in supabase
  • navigate to /storages route in coolify dashboard
  • add supabase s3 bucket details (bucket name, region, endpoint, secret key, access key)
  • submit
@ukashazia ukashazia changed the title [Bug]: Error adding Supabase S3 storage to "S3 Storages" [Feat]: Add Supabase S3 storage to "S3 Storages" May 2, 2024
@luckycreationsindia
Copy link

Isn't it working?

I tried using R2 Storage using S3 by changing endpoint when adding storage details and it's working without any issues.

If I'm not wrong supabase is compatible with S3 Client. So it should work.

Also when I looked into supabase docs I found that it's not fully implemented but some commonly used endpoints will work.

Reference:
https://supabase.com/docs/guides/storage/s3/compatibility

@ukashazia
Copy link
Author

ukashazia commented May 3, 2024

Isn't it working?

I tried using R2 Storage using S3 by changing endpoint when adding storage details and it's working without any issues.

If I'm not wrong supabase is compatible with S3 Client. So it should work.

Also when I looked into supabase docs I found that it's not fully implemented but some commonly used endpoints will work.

Reference: https://supabase.com/docs/guides/storage/s3/compatibility

According to Coolify's S3 docs, R2 is supported but in the list of compatible S3 providers, there's no supabase S3. I tried linking Supabase's S3 bucket but there seems to be an issue with how the URI is built from the S3 details provided to coolify.

@ukashazia
Copy link
Author

ukashazia commented May 3, 2024

Screenshot for reference:

Screenshot_20240502_145414

There seems to be a duplication in url string which should be the reason for the 404.

@luckycreationsindia
Copy link

Can you share endpoint URL (hiding your project reference from URL). Maybe the Issue is with the endpoint you are providing.

As per the doc it should work.

I just checked that my region is auto (In coolify S3 Settings) when connecting with R2. Can you also try that in supabase endpoint.

And yes it's duplicating the URL string. Make sure you add endpoint as https://[project-ref].supabase.co

This should work.

@ukashazia
Copy link
Author

ukashazia commented May 3, 2024

Can you share endpoint URL (hiding your project reference from URL). Maybe the Issue is with the endpoint you are providing.

As per the doc it should work.

I just checked that my region is auto (In coolify S3 Settings) when connecting with R2. Can you also try that in supabase endpoint.

And yes it's duplicating the URL string. Make sure you add endpoint as https://[project-ref].supabase.co

This should work.

Removing /storage/v1/s3 from endpoint completely removes it from the resultant URI which throws 404 again.

@luckycreationsindia
Copy link

Can you share screenshot of error after removing /storage/v1/s3 from endpoint

Here's my setting for R2 (Can't say about supabase)

Endpoint - https://[account-id].r2.cloudflarestorage.com
Bucket - xxx
Region - auto

@ukashazia
Copy link
Author

Can you share screenshot of error after removing /storage/v1/s3 from endpoint

Here's my setting for R2 (Can't say about supabase)

Endpoint - https://[account-id].r2.cloudflarestorage.com Bucket - xxx Region - auto

Sorry if i didn't mention that earlier. I have my R2 setup successful but that's not the case with supabase.

@luckycreationsindia
Copy link

I tried supabase. There's actually a bug in coolify which is appending anything after slash two times. Maybe this issue is with underlying S3 Client library in coolify.

@ukashazia
Copy link
Author

I tried supabase. There's actually a bug in coolify which is appending anything after slash two times. Maybe this issue is with underlying S3 Client library in coolify.

Anyways, i find R2 much better for my current usecase

@luckycreationsindia
Copy link

I tried supabase. There's actually a bug in coolify which is appending anything after slash two times. Maybe this issue is with underlying S3 Client library in coolify.

Anyways, i find R2 much better for my current usecase

Ok. But still let keep this issue open if anyone wants to use supabase or any other service where endpoint has path included in URL

@ukashazia
Copy link
Author

I tried supabase. There's actually a bug in coolify which is appending anything after slash two times. Maybe this issue is with underlying S3 Client library in coolify.

Anyways, i find R2 much better for my current usecase

Ok. But still let keep this issue open if anyone wants to use supabase or any other service where endpoint has path included in URL

i might as well contributed and fixed it it was written in js. looks like it's implemented in PHP :)

@luckycreationsindia
Copy link

I tried supabase. There's actually a bug in coolify which is appending anything after slash two times. Maybe this issue is with underlying S3 Client library in coolify.

Anyways, i find R2 much better for my current usecase

Ok. But still let keep this issue open if anyone wants to use supabase or any other service where endpoint has path included in URL

i might as well contributed and fixed it it was written in js. looks like it's implemented in PHP :)

Same here. I tried checking the code it's using "flysystem-aws-s3-v3" library. Checked the code where it's concatenating the endpoint with bucket. Couldn't find more as I don't know how PHP with Laravel actually works. I just assumed it's issue with the library.

@actraiser
Copy link

actraiser commented May 4, 2024

Out of curiosity, how do you configure a Supabase S3 Storage in the UI? I use R2 with Supabase but what I did is to hardcode the R2-Endpoint, Bucket and Region in the docker-compose.yml under the supabase-storage-service and changing Minio User/Password in the Coolify UI to reflect the credentials on R2.

    environment:
      - STORAGE_BACKEND=s3
      - STORAGE_S3_BUCKET=[R2 bucket]
      - STORAGE_S3_ENDPOINT=[R2 endpoint] # replaces http://supabase-minio:9000
      - STORAGE_S3_REGION=[R2 region]

Is there a simpler way using the Coolify or Supabase UI directly?

Greets
-act

@ukashazia
Copy link
Author

ukashazia commented May 4, 2024

Out of curiosity, how do you configure a Supabase S3 Storage in the UI?

You can add an S3 bucket in /storages. But this bucket can only be used by coolify to store backups of databases as far as i know.
After you've successfully added a bucket, you have to select it from backups section/screen of a database to backup your db backups to S3.
.

@actraiser
Copy link

actraiser commented May 4, 2024

You can add an S3 bucket in /storages. But this bucket can only be used by coolify to store backups of databases as far as i know.

Ah sorry, i misunderstood - you are talking about database backups to S3 and not about using an external S3 Bucket for the Supabase Storage API (to have an external CDN). You can actually easily use the Coolify configured S3 Storage for Supabase database backups by going to your Supabase resource, then scroll all the way down to the supabase-db service in the list and click on settings. there is a "backups" option that you can configure to schedule backups to your S3 buckets you have configured with Coolify before. At least for me - R2 works totally fine.

Bildschirmfoto 2024-05-04 um 08 17 45

And it will show up in R2:

Bildschirmfoto 2024-05-04 um 08 21 22

Greets
-act

@ukashazia ukashazia changed the title [Feat]: Add Supabase S3 storage to "S3 Storages" [Feat]: Add Supabase as S3 provider for Coolify May 4, 2024
@ukashazia
Copy link
Author

We were trying to add S3 as a db backup location to coolify. R2 works but supabase's hosted/managed S3 (not the selfhosted one) is having some issues.

@luckycreationsindia
Copy link

As per document I think supabase is not tested and is not supported.

https://coolify.io/docs/knowledge-base/s3-storages

@andrasbacsai andrasbacsai self-assigned this May 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants