Security overview

In one page

DataPulse AI is a Databricks App that runs inside your workspace. We host nothing, receive nothing, and have no access to your account.
Zero egress. The app calls no external service and needs no API key. The only model it calls is a Model Serving endpoint in your workspace. Notifications go only to your own Databricks notification destination.
Your data is read as the signed-in user (on-behalf-of). Unity Catalog permissions, row filters and column masks apply. The app cannot see more than the user can.
The app's own service principal writes the app's state (an action ledger and an audit log) and performs approved actions. It has no grant on your data.
Every change is an action: preview, approval by role and risk, execute, verify, roll back, all recorded in an append-only ledger. If the ledger cannot be written, the action does not run.
No stored credentials. No personal access tokens. The user's short-lived token is never logged or stored.
Licence check is offline. No phone-home, no telemetry.

Not certified. There is no SOC 2 report yet; it is on the roadmap. The design follows least privilege and zero egress so your own review can be done from this page, the full security overview we send with each release, and the code.

Architecture

Everything runs inside your Databricks account, in one region.

Component	What it is	Runs as
React UI, Express API	The app, in Databricks Apps serverless compute	The app's service principal, plus the user's token per request
SQL warehouse	Your serverless warehouse, bound as a resource with `CAN_USE`	The caller (data) or the service principal (state)
`datapulse_app.state`	Delta tables in Unity Catalog. No separate database	Written by the service principal
Model Serving endpoint	An endpoint you choose, bound with `CAN_QUERY`	The service principal
Platform jobs	`recipe_runner`, `finops_audit`, `guardrail_check`, `health_check`, bound with `CAN_MANAGE_RUN`. Schedules ship paused	The bundle owner

Identities: the user or the service principal

Which identity each operation uses.

Operation	Identity	Why
Read `system.*` and your tables, profile, dry-run (at most 100 rows), run a cleansing recipe	User (OBO)	Unity Catalog permissions, row filters and masks apply. The app never has more access than the user
Identify the caller and check group membership	User (OBO)	The identity comes from the warehouse (`current_user()`), never from a request header
Read and write who may do what (`user_entitlements`)	Service principal	Users cannot edit their own access; only the API, for an admin, writes it
Write the action ledger, audit log, budgets, SLAs, rules, incidents and recipe history	Service principal	Users cannot edit the audit trail
Model Serving calls	Service principal	The endpoint is an app resource
Start a bundle job	Service principal, after a user pre-check	The pre-check confirms the requesting user could do the same work. If it fails or cannot be decided, no job starts
Approved actions on your resources	Service principal, after approval by the right role	Auditable and reversible

The app asks for one user scope, sql. Access has three layers: the app's CAN_USE group decides who may open it, a Delta table of entitlements decides what they can do (roles reader, cleansing engineer, operator and admin), and Unity Catalog decides which data they see. A role never adds data access. Every change to a role writes an audit row first, and the API always keeps at least one admin user.

What the service principal can do

It has no grant on your catalogs or tables. Beyond the app's own state schema, its warehouse, its model endpoint and its four jobs, it needs a per-resource permission only for the actions you want DataPulse to take, and you grant it only on the resources you want managed. With none granted, the app still reads, reports and previews; an approved action on an ungranted resource fails visibly and is recorded as failed.

Action	Permission	Risk	Approval	Rollback
`stop_warehouse`	`CAN_MANAGE` on that warehouse (it also lets the app see every user's queries on it, which the idle check needs)	Low only when no query is running or queued, else high; re-checked at execute	Operator (low) / admin (high)	Start it
`start_warehouse`	`CAN_MANAGE` on that warehouse	Low	Operator	Stop it
`stop_app`	`CAN_MANAGE` on that app	High	Admin	Start it
`terminate_idle_cluster`	`CAN_RESTART` on that cluster	High (idleness cannot be proven live)	Admin	Start it (configuration kept)
`set_auto_termination`	`CAN_MANAGE` on that cluster	Low	Operator	Previous value
`resize_cluster`	`CAN_MANAGE` on that cluster	High	Admin	Previous specification
`pause_job_schedule`	`CAN_MANAGE` on that job	High	Admin	Unpause
`set_job_timeout`, `set_job_retries`	`CAN_MANAGE` on that job	High	Admin	Previous settings
`repair_run`	`CAN_MANAGE_RUN` on that job	Low	Operator	Cancel the repair while it runs; tasks already re-run stay re-run
`retry_pipeline_update`	`CAN_MANAGE` on that pipeline, and the caller must see the pipeline	Low	Operator	None; the dialog says so
`full_refresh_pipeline_table`	`CAN_MANAGE` on that pipeline, and the caller must see the pipeline	High	Admin	None; the dialog says so
`productionize_recipe`	State schema only (writes a schedule row)	High	Requested by a cleansing engineer or operator, approved by an admin	Recorded; the schedule is not yet disabled automatically (see known limits)

Actions never run silently: the default mode is approval. Only an Auto-Idle budget (a Business-tier setting) runs low-risk actions on idle resources without a person, and each is still recorded and reversible. Idle is proved from live query history and fails closed: if the history cannot be read, the action is high risk, and if the warehouse has turned busy by the time it executes, the action goes back to an admin and nothing runs.

Data flows

The read path, the change path and the model path.

Reads go from the warehouse to the API to the user's browser. Nothing is copied to another store, except aggregates the app keeps in its state (for example an incident record).
Writes to your data are limited to one thing: a cleansing recipe's CREATE TABLE AS SELECT (or CREATE VIEW) into the target schema, <catalog>.silver.<table>_cleaned by default. A run replaces only a table DataPulse created. The app never updates, deletes or drops your tables.
Operations and governance statements are built from fixed templates with validated identifiers. A model never writes them.
State lives in datapulse_app.state in your catalog, under your retention rules: the ledger, audit log, recipes and schedules, budgets, SLAs, rules, incidents, settings and entitlements. No tokens and no row data.

Zero egress

No external SaaS, database or API key. No calls to us. The UI is built into the release and served by the app itself.
Which models may run is your choice: the built-in endpoint, or the allow-list an admin sets in Setup → Models. Any other model is refused, and if the allow-list cannot be read, no model is called at all.
Notifications go only to a Databricks notification destination you configure. For Microsoft Teams, DataPulse creates a paused Databricks SQL alert over its own state; Databricks, not DataPulse, delivers the message. The app never stores or calls a Teams webhook.
The licence is verified offline. The annual true-up report is generated from your system.billing data, shown to you, and shared by you. Nothing is sent automatically.
We have no network path into your workspace. Support is by screen share or by logs you choose to export.

AI and model data handling

The model is used in three product features: SQL generation for Data Cleansing, the Query Tuning optimizer, and pipeline root cause. Our own evaluation runs before a release use a test workspace, never yours.

Feature	What is sent to the endpoint in your workspace
Cleansing SQL	Table and column names, types, null and distinct counts, and the user's typed instruction. While Share sample values with the model is on (the default), also up to 5 sample values for each column with 20 or fewer distinct values
Query optimizer	The text of the slow SQL statement and its metrics
Root cause	The error text of a failed run and job metadata

Not sent: raw table rows, columns with many distinct values, credentials, tokens, or anything from other users' sessions. Sample values are real values from low-cardinality columns and are not masked. An app admin can turn them off; the change is confirmed and audited, and if the setting cannot be read, no sample values are sent.

Inputs from your tables (column comments, sample values, error text, SQL text) are wrapped in delimiters and passed as data, not instructions.
Output must match a strict schema, with one repair retry; nothing unvalidated reaches the UI.
Generated SQL passes the safety filter; a blocked statement is never executed. AI output is labelled, and every estimate says it is an estimate.
Prompt-injection cases are part of the release evaluations, and a model change must pass them.
Because the endpoint is in your workspace, the model provider terms are those of your Databricks contract, not ours.

Audit ledger

PreviewThe change, its risk, the downstream impact and how to roll it back.
ApproveBy role and risk: an operator (or a cleansing engineer, for Data Cleansing) for low risk, an admin for high risk.
ExecuteRecorded in the ledger first. If the ledger cannot be written, nothing runs.
VerifyAfter the observation window, a person presses Verify and the effect is measured.
Roll backRestores the previous setting. Retry update and Full refresh cannot be undone, and the dialog says so.

The actions table is an append-only Delta table. Every state change of an action is a new row with the actor's email and role, the target, the risk, the preview, the baseline, the result and the rollback plan. Nothing is updated in place.
The workspace_audit_log is append-only too.
Fail closed. If a ledger or audit write fails, the action does not run.
An optional change window limits when approved actions can execute.
Realized savings count only actions in the verified state, so the Overview cannot claim a saving that was not measured.
You own the tables: query, export and retain them like any other Delta table.

What we never access

We have no login, token or network path to your workspace or account. The app does not report to us.
No customer row data is transmitted anywhere by the app.
No personal access tokens are created, stored or requested. The user's token is used for the request only.
No access to your tables by the service principal.
No writes to your data other than a cleansing recipe's new table or view, and no deletes.
No hidden accounts. The service principal exists because the platform creates one for every app, and you can inspect or revoke each of its permissions.
No usage metering. We cannot see how the app is used.

Known limits in this release

Stated so a reviewer does not assume more than exists.

Verification and the platform jobs are thin. The three scheduled jobs do not yet do the checks the app's own buttons do, and nothing moves an action from executed to verified on a timer. Keep schedules paused.
A failing job opens an incident only when the health_check job runs, and its schedule ships paused. Incidents also come from SLA breaches, recipe expectation failures, and Diagnose with AI on a failed pipeline update.
Sample values are not masked (see above). An admin can turn them off; they are on by default.
Rolling back a productionized recipe is recorded but does not yet disable its schedule.
The licence workspace count is reported, not enforced.
Not certified. There is no SOC 2 report yet.

Requirements

Unity Catalog with system tables enabled, serverless compute, a serverless SQL warehouse, a Model Serving endpoint you can query, and the grants in the admin guide. One install covers one account and one region.

Dependencies and the full document

Before each client release we run npm audit on a clean install. The security overview we deliver with the release lists every advisory that remains and whether it is reachable. Ask us for it, and for the admin guide's full list of grants, at connect@raziconsulting.com.

Reporting a vulnerability

Email connect@raziconsulting.com with "Security" in the subject. Do not include customer data. Response commitments for customers are set in their contract.