My platform expects multiple writes from various users belonging to one organization.
What's the best way to serialize the writes? Scale is 50 to 100 users wanting to write around the same time.
i came across a duckdb extension which accepts firestore as the source and then we can run all duckdb ops in sql.
want to check if anyone have tried it.
i came across a duckdb extension which accepts firestore as the source and then we can run all duckdb ops in sql.
want to check if anyone have tried it.
hello community, i am looking to find my CTO co-founder.
i am building two products. One is an ecosystem product similar to Salesforce/Zoho/Odoo. another one is a cloud tech stack that would compensate for the lack of technical capabilities of dev teams. There's so much it. This cloud tech stack is to power the ecosystem product as we hit the scale. I will explain at depth when we connect what I mean by this.
i am fairly technical. so far, im the one playing all roles and i want another brain to be there in building alongside me.
i usually carry a perspective that cofounder level relations cannot be just "found" over a post. its a dynamic that you build. but nonetheless, i want to give this a try and see if i find any potential partner.
current stage: product is live with early customers
team: 5 members including myself
you would be surprised to see the kind and the number of things we have built in last few years and where we are today.
i am expecting my cto cofounder to be intelligent, problem solver, good at engineering n planning, have leadership abilities, can operate in uncertainty, understands cost, competent to build scale.
and of course the personality match. that we can only know until we catchup.
All the reads and writes happen locally. The writes get synced to the server side sqlite. The websocket alerts the other clients that are online that belong to the same tenant. Then the clients pull the latest changes from the server side db. Instead of crtd, I chose event based sync.
This is the crux. Each tenant gets it's own db. But I'm wondering what all I need to take care of.
Any suggestions on sync, or stress testing or making it HA etc?
My product has grown out of Firestore in terms of cost. The weekly firestore backups itself is costing me more than the firestore costs. To circumvent around the limitations of Firestore native mode standard edition, we have used RTDB, Supabase and recently added Firestore native mode enterprise edition.
The costs are becoming painfully bigger. Especially from RTDB and Firestore's backup.
I requested google for startups team over mail for some extra credits for 3 to 6 months but faced disappointment. I honestly was hoping for a better response. For years, I have submitted so many vulnerabilities to Google under their VRP, and even held 75th as my best rank at some point.
I don't know if it was due to this attachment, or what, I was disheartened for the lack of short-term support. The time I spent on hunting for vulnerabilities in Google products back in the day, the rewards I have received from them suddenly felt meaningless.
I question myself if I have asked too much. As an early stage growing startup that's full of potential, I would give them great business in coming months. But now, due to the costs, I am having cloud bill anxiety. I cannot ask my existing customers to not use the product. However until I figure out things, I have halted onboarding any new customers.
Now, I don't know what divine timing, I have been getting constant calls from AWS partners with the promise of (way) cheaper costs and completely free migration solution. One of the partner company CTO is top of the line. The solution he proposed is too good. But again, I would be stuck in AWS ecosystem.
I don't want to jump from one serverless stack to another serverless stack.
Around the same time, I got a call from GCP Business Development team asking for my experience of GCP. Later into the call I have realised it was more of a sales call than anything.
If I'd promised them to make a billing of $25,000 USD on Vertex within 6 months, she would give credits commitment.
Honestly, the call made me repellent from GCP further. I wish at least they offered me a call with solution architect who could take a look at our billing dashboard and help us reduce our costs.
Anyway, since then, I am now obsessively exploring a cheaper stack that I can thoughtfully build. This time, the advantage we have is the domain knowledge, how all things are connected, the overall bird eye picture, the limitations we faced etc.
So right now I am leaning towards running SQLite at both server side and client side for transactions, tantivy for search, chromadb for AI, all on one server, backups on GCS, GCS powered datalake.
Started building a sync engine with web sockets for real-time support and cleared almost all the use-cases and stress tested it with 200GB of real business data per tenant. Its all working fine. Now, need to handle the edge cases.
Since its local, the response time is immediate.
The whole experience started pushing me to explore local first stack, research into Figma's realtime sync engines, how notion, linear, slack all are working under the hood. Not sure how long the migration will take now. Our internal estimate are 3 months.
Not expecting anything from this post. Just a rant as a Google's/Firebase lover and advocate who is about to part ways. If I am not, I wouldn't have bothered.