DataPulse AI user guide
DataPulse AI runs inside your own Databricks account. It shows where your platform money goes, helps you fix bad data, and watches your pipelines. It never sends your data outside Databricks.
This guide is for data leads and FinOps managers. It uses the same words you see on screen. If you run the install, read the admin guide instead.
New to DataPulse? Start with the Application Manual — one document covering every page, with step-by-step walkthroughs and a glossary.
Where to start#
| If you want to… | Read |
|---|---|
| Open the app and find your way around | Getting started |
| See the money saved and the top risks in one page | Overview |
| Find waste, split cost by team, set budgets | Spend |
| Speed up slow queries with AI | Query Tuning |
| Clean a table and turn the fix into a scheduled job | Data Cleansing |
| Find failing jobs, missed SLAs, and their cost | Pipelines |
| Approve, run, and undo changes | Actions and approvals |
| See every background job the app started, and its schedule | App Jobs |
| Know what your role can do | Roles |
| Get a quick answer | FAQ |
| Fix an error message | Troubleshooting |
The seven pages and their names on screen#
The left navigation has seven entries.
| Guide page | Navigation label | Job in one line |
|---|---|---|
| Overview | Overview | Proof of value — KPIs, risks, approvals |
| Spend | Spend | Know where every DBU goes and stop waste |
| Query Tuning | Query Tuning | Find slow queries and rewrite them with AI |
| Data Cleansing | Data Cleansing | Fix bad data with plain-English requests |
| Pipelines | Pipelines | One board for every job, SLA, and incident |
| App Jobs | App Jobs | Background work the app started and recipe schedules |
| Setup | Setup | Health checks and settings (admins) |
Data Cleansing is hidden below the Team licence tier.
Three rules that hold everywhere#
- Nothing changes without a preview. Every change to your platform is an action. You see the change, its risk, and how to undo it before you confirm. See Actions and approvals.
- You see only what you may see. DataPulse reads your data with your own identity, so Unity Catalog permissions apply. A table you cannot read is not listed.
- Every number can be traced. Look for Source, How calculated, Why, or Lineage links next to a number. They open the query or table behind it. Numbers from an AI model are labelled AI estimate.
What the numbers are and are not#
- Mock Data or Mock Timeline in a data-source strip means the panel shows sample data, not your bill. See Spend.
- Money saved counts as verified only after an action has been confirmed by a follow-up measurement. Estimates are never added to verified savings.
Known limitations in this release#
| Area | What to expect | Open item |
|---|---|---|
| Verified savings | An executed action becomes verified only after the observation window (DATAPULSE_VERIFY_DAYS, default 7 days) has passed and a person who may change that action's module presses Verify on it in the Actions inbox. Nothing verifies it automatically. Until then Verified Savings on the Overview does not count it. | none (by design) |
| Rightsizing and idle waste | These read system.compute.node_timeline and system.compute.clusters. A workspace with only serverless compute has no classic clusters, so the panels have nothing to show. | OI-169, OI-200 |
| Compute menu | The list of SQL warehouses can read No warehouses available when the app cannot read a warehouse's permissions. Start on a stopped warehouse needs the app's service principal to hold CAN_MANAGE on it. | OI-221 |
| Pipelines job settings | For a job the app does not own, the hygiene checks can stay unchecked, and the message says no grant changes it. A CAN_VIEW grant to the app's service principal on that job is the likely fix; this has not been tried live. | OI-237 |
| Data Cleansing profile | A profile of a table over 10,000 rows uses a 500-row sample and chooses checks by column name. | OI-191, OI-192 |
| Spend anomalies | The list can be long, and its Expected column mixes several workspaces and tags. Treat it as a list of leads. | OI-219 |
| Roles seen by a second person | The role limits are enforced and tested, but they were checked live with one signed-in user only. | OI-30 |
About the screenshots#
The pictures were captured with sample data at 1440 pixels wide. Numbers in them are examples. Numbered orange badges match the list under each picture. To rebuild the pictures after a UI change, run npm run docs:guide-screenshots (it calls npx tsx scripts/capture_guide_screenshots.ts). That script uses mock data only, so it costs no Databricks credit, and it stops with an error if a labelled element has moved.