The pitch
We're an AI-native company. Smart people, creative thinkers, always looking for ways to use AI to work better. Someone raised a fair question: why are we paying for a BI tool when an AI agent can generate dashboards from code in minutes?
And honestly? The reasoning was good. Static dashboards for standard reporting. Something interactive for the rest. An AI agent could spin one up faster than anyone could click through a drag-and-drop builder. Schema changes in the warehouse? The agent catches them and fixes the downstream reports automatically. Version control, code review, all the good engineering practices - baked right in.
Everyone was nodding. Great idea. Let's do it.
I was the one who had to say hold on.
What the idea was missing
The first thing that hit me was observability. Our BI tool is a control tower. I can search across every dashboard in the company. I can see what's being queried, who's querying it, how often. That sounds boring until you need it. (You always need it.)
With code-based dashboards, sure, you can build search. You can build usage tracking. But now you're building and maintaining stuff that already exists in a tool you're paying for. You're trading a solved problem for a new project.
Then there's access control. Our BI tool handles role-based permissions at the company level. Anyone can manage it - technical or not. It's visible. It's out in the open.
With custom dashboards, you'd probably push access control down to the warehouse level. Fine-grained table permissions, row-level security. That works, but it's a black box. Harder to audit, harder to manage, and harder for non-technical people to understand. We're dealing with customer PII here. That's not where I want to get creative.
Security patches, maintenance, uptime - the BI tool handles all of that too. Custom dashboards? That's on us now.
The two-tools problem
The next suggestion was fair: what about both? Custom dashboards for some things, the BI tool for the rest.
I've been down this road. It sounds clean in theory. In practice, you end up with a question nobody can answer clearly: when do we use which one?
Custom dashboards are for X, Y, and Z. The BI tool is for A, B, and C. Somebody has to know that. Every new person who joins has to learn that. And without the line being really, really obvious - people guess wrong. They build the thing in the wrong place. They look for a report and can't find it because it lives in the other system. They duplicate work because they didn't know it already existed somewhere else.
You've made your end users more confused, not less. The whole point of a data team is to make data easier to use.
This isn't really about BI tools
Here's the bigger thing. AI is making the tooling problem feel like the whole problem. It's not.
Data has never been a tooling problem. The hard part is understanding - the data, how the business works, the processes that generate the numbers. Which deals count as pipeline? What does "active" mean for your product? When finance says revenue, do they mean the same thing sales means?
AI gets you to the dashboard faster. That's great. But it doesn't get you to the understanding faster. You can spin something up in five minutes, but if you don't know what the numbers actually mean - how the business uses them, what the edge cases are - you've just built a faster path to the wrong answer.
The tools are easier to access now. The understanding isn't. I think we're going to see a lot more of this. Teams moving fast with AI, then wondering why the data still doesn't feel right.
Saying no is part of the job
I loved the idea. I said that at the time. The thinking was creative, and the instinct to question existing tools is exactly right.
But my job is to hold the whole picture. Not just "can we build this?" but "should we?" What do we lose? What new problems show up? Who gets confused? What breaks when the person who built it moves on?
At an AI-native company, there's a pull toward replacing everything with something custom. Something code-first. Something the AI can touch. And sometimes that's the right call. But sometimes the boring tool is boring because it already solved problems you don't want to re-solve.
Good data work isn't about saying yes to every clever idea. It's knowing when to push back - kindly, with reasoning, without killing the energy.
The hardest part of AI adoption isn't what to build. It's what not to.