u/CommitteeImmediate66

For the last few months I've been building out our core dashboards directly in Databricks AI/BI (their Lakeview dashboards) instead of piping everything into a separate BI tool.

My findings/highlights have been:

- The dashboards sit right on top of our lakehouse tables, so there's no extract/import/refresh dance. What's in the warehouse is what's on the dashboard. That alone killed a whole category of "why don't the numbers match" tickets.

- Permissions, lineage, and the underlying tables all live under the same Unity Catalog governance. I'm not maintaining a separate security model in the BI tool. We're on azure so it's easy to sync entra groups.

- Genie for the long tail of ad-hoc questions. This is the part I didn't expect to like as much as I do. Instead of building (and then maintaining) 40 variations of the same dashboard for every stakeholder's "but can you also show me..." request, I stand up a Genie space on top of the same curated tables. Business users just ask questions in natural language and get back charts on the fly. This has cut my ad-hoc request backlog dramatically and the business is pretty happy with response quality.

The one downside I've noticed is the visualization/formatting options are sometimes limited, but not a major blocker.

Here's my actual question for the sub: some of my colleagues still lean toward Power BI by default, even when the data already lives in Databricks. I get the ecosystem/familiarity argument, but I'm trying to understand the reasoning beyond "it's what we've always used." For those of you who'd still pick Power BI (or Tableau/Looker/etc.) over building natively in the platform where your data sits - what's driving that? Is it the better viz customization capabilities, the semantic model, self-service maturity, org politics, something else?

Genuinely trying to pressure-test my own enthusiasm here, so push back if you think I'm missing something.

I have a background in data science but am relatively new to databases and apps which I'm trying to upskill on as I'm building more LLM based agents. I work on Databricks mostly and came across the Databricks docs (link in comments) suggesting using lakebase for storing agent state.

I'm curious to hear people's experiences and also build a mental model of understanding when to consider which kind of database e.g. Managed postgres, Vs standard delta tables vs. even an unstructured data store such as redis for storing state. Are there any pros/cons in terms of user auth etc. That I should be aware of?