u/himalgnn

For those managing large datasets, how are you deciding what stays on-prem and what goes into public cloud?

We're considering an architecture where frequently accessed data stays locally while older or less frequently accessed data gets moved to cloud storage. The appeal is obviously cost and flexibility, but I'm wondering how complicated this becomes operationally.

For anyone who has implemented something similar, does this actually simplify things, or does hybrid storage just create another layer of complexity?

Would you recommend building around S3-compatible object storage, or would you keep things simpler with traditional NAS plus cloud backup?

We're reviewing our backup architecture and one thing I'm particularly interested in is having an immutable copy that isn't easy to compromise if the production environment gets hit by ransomware.

Right now we're comparing dedicated backup appliances, immutable object storage, and cloud-based approaches.

I've seen Cloudian come up quite a bit because HyperStore supports S3 Object Lock/WORM-style retention and can be used as an on-prem backup target.

For people who have actually implemented immutable object storage, what has your experience been like?

Is object storage a good backup target compared with a dedicated appliance? And how much operational overhead is involved in maintaining the storage cluster?

Every vendor in this space says "100% S3 API compatible" and every vendor turns out to have some edge case that breaks a tool you actually use. Wondering what people have hit in practice.

We tested Cloudian HyperStore against our existing S3-based backup tooling (Veeam and a couple of internal scripts using boto3) and didn't hit anything that needed workarounds, which was honestly a bit surprising given past experience with other "S3-compatible" platforms. Also tested MinIO in the same environment, similar result on compatibility, the differences showed up more in tiering and multi-tenancy features than the API itself.

Has anyone run into real compatibility gaps with any of the major players that weren't obvious until production traffic hit it?

One thing I've noticed while researching enterprise storage is that most discussions focus on today's capacity, but very few talk about what happens three or five years later when the environment has doubled or tripled in size.

That made me start looking more closely at scale out object storage architectures instead of traditional storage systems.

For people who have managed storage growth over several years, what decisions ended up saving you the most time and money? Was it the platform itself, the architecture, automation, or something completely different ??

I am really curious what everyone is using for object storage at scale. For teams managing TBs or PBs of data, are you mostly using public cloud storage, self-hosted solutions, or a hybrid approach?

What are the biggest issues you’ve run into : cost, performance, management, reliability?