u/Kingkif

I have been playing around with DuckDB and it is fantastic for local data wrangling and small analytical tasks. However, some engineers on my team are pushing to use it as the core analytical engine for a new customer-facing reporting feature.

I am highly skeptical. It is an embedded database, not natively distributed. How are we supposed to scale this when we hit terabytes of data? Are we just going to end up building a complex distributed system around an embedded database? I have also read concerning reports about memory corruption and performance issues with complex pivoting on large datasets. Is DuckDB actually viable for critical, large-scale production analytics?

We are evaluating engineering analytics tools and Waydev is on the shortlist. However, I have been reading some reviews that say the initial setup and onboarding is incredibly complex and intimidating.

Apparently, it requires days of debugging, complex ETL processes, and data mapping just to get basic dashboards working. We do not have a dedicated data engineering team to babysit this tool. We need something that works out of the box and gives us visibility from the team level down to individual PRs without requiring a PhD to configure. Has anyone used Waydev recently? Is the setup really that bad?

I’ve built a mock application using AI with dummy data, but I’m now stuck on how to turn it into a production-grade application. I’m looking for guidance on the steps, best practices, or key areas I should focus on to make it ready for real-world use. Any help would be appreciated.