How much work is it to run a personal private yellowstone grpc ?
Hey,
I have some insane use case where it could make sense for data purposes just for me, to rent a datscenter rack and put a powerful 32-64 cores cpu 512gb ecc ram type stuff server in there. And yeah. Run my own solana data pipeline.
I want to do sophisticated onchain trading, insane onchain wallet tracking/monitoring.. clustering smart/dumb money.. not just monitoring a few wallets.. keeping an eye on basically the whole chain and everyone.. and keeping an eye on all tokens and all wallets.. all transactions except for like the validator stuff i don't care about. So yeah I think it's gonna require an insane amount of data.. I'll be doing like tree models of entities.. entity clusters.. who sent solana/token to whom.. who might be an insider.. how much smart (profitable traders) currently in this coin versus dumb.. which wallets or entities(clusters) tend to bring liquidity to the coin after they buy.. just basically stuff I can't do with like 50usd/mo RPC subscription or something like that, not even with the cheaper grpc subscriptions.. just a toon of data.
I was wondering, how much work/maintenance is it once its up and running, to run a yellowstone grpc ? How often do you have to check your yellowstone grpc ? For personal use case, not sharing publicly.
I might want to run it unmetered, for constant data streams of a ton of data so its starting to look like a better investment for me rather than pay crazy expenses every month and not own it. If i own it I can do various stuff on there without even having to transfer some of the processed data to my network right. So yeah I might want to do it. I could also co-locate directly in that rack there right. The datacenter would cost me a bit, but i already got the costs of everything calculated so now it's just about whether I'll actually go for it or no. Which I guess i'll see, still considering it.
But yeah.. what do you need to keep an eye on when running it ? What tends to break ? How often do you have to update or do restarts ? And just how difficult is it to deploy it ? I mean I understand linux and computers generally, i can code.. + i got claude code so i guess I'm chill ? A few days of work and chill ? And if there are issues or stuff that's problematic, is it hard to build automations around it to make it less of a headache so you can almost forget it for months at a time and have it just work fine ? Most of the time ? Like i don't mind that much if it breaks once every few months and then it's fixed within day and boom forget it again for like 3-6 months.. is it like that ? Can it be ?
Yeah, if you have any experience or any advice then I'm definitely curious, anything i should watchout for ?
Also question, in practice what hardware/specs you think are truly good to have here ? I was thinking put a powerful 32-64 core cpu, 512gb ddr4 ecc ram with option to expand to 1tb in the future if needed. And internet connection 3gbps but expandsble to 10gbps if i request the datacenter.
And start with 2TB commercial ssd drive for the ledger itself, and whatever data i decide to store would go compressed to my data server which has over 100TB of free space currently. Do i need more than 2tb on the RPC itself ?
Would you say its overkill ? Or just right ? Or would you actually recommend more power ?