NATS Object Store as Source and Publisher storage

[HeCorr]

Since this project is now embracing NATS I believe it would be quite interesting to see added support for NATS Object Store, eliminating the need to setup an S3 server which could greatly simplify small projects that use Bacalhau.

However, keep in mind that Object Store is still in Experimental Preview.

[wdbaruni]

Hey @HeCorr, thanks for the suggestion. How do you see NATS Object Store fit in a bacalhau network?

Today we are embedding NATS servers within the requester/orchestrator nodes. An option would be for the requesters to host the object store, but I'll be concerned about adding an additional and heavy responsibility to the orchestrators

Another option would be to add a separate cluster of NATS servers to form their own RAFT group to replicate and store the objects. Kind of a super cluster of two cluster types, one for orchestration and others for storage. Though I am not sure if this can actually work and if NATS will route get and put requests across clusters.

What are your thoughts?