RaziConsulting Services
Menu

DataPulse AI / Data Cleansing

Data Cleansing

Turn a raw bronze table into a clean silver table without writing the SQL. Describe the fix in plain English, review the SQL behind a safety gate, preview 100 rows, then write a new table. Your source table is never changed.

Book a demoRead the Data Cleansing guide
Seven steps

From a messy table to a scheduled, checked pipeline.

1 · PROFILE

Profile

A health score out of 100 with its reasons, null and distinct counts, defects such as numbers stored as text, and Databricks' own data-quality monitor status when one is enabled.

2 · RULES

Rules and scorecard

Not null, numeric range, pattern, unique and foreign-key rules, each with rows passed and failed, run with your own access.

3–4 · DESCRIBE

Describe, then review the SQL

Type the fix in plain English. A model your admin allows writes plain ANSI SQL, and the safety gate checks it before you see it.

5–7 · RUN

Dry-run, run, productionize

Preview 100 rows, write the silver table now or as a background job, then schedule it with enforced expectations after an admin approves.

Safety gate

Nothing destructive runs. Your source stays read-only.

  • Generated SQL passes a safety filter that blocks DROP, DELETE, UPDATE, INSERT, ALTER, GRANT and multi-statement input. A blocked statement is never executed.
  • A request to drop or delete the table is refused before any model is called.
  • The only write to your data is a CREATE TABLE AS SELECT (or a view) into your target schema, silver by default. A run replaces only a table DataPulse created.
  • Runs use your own permissions. A background job starts only after a check that you can read the source and write the target.
The Review SQL step: safety gate passed, the source table marked read-only, the target silver table, and the generated SQL. Numbered markers label the parts.
Review SQL · sample data. Markers: 1 safety gate passed, 2 source read-only and a new silver target, 3 the generated SQL, 4 next, a 100-row dry-run.
What the model sees

Metadata and statistics, not your rows.

The model receives table and column names, types, null and distinct counts, and your typed instruction. By default it also receives up to 5 real, unmasked sample values from each column with 20 or fewer distinct values, because a mapping such as 'SHPD' → 'Shipped' needs the real categories.

  • An admin can turn sample values off in Setup → Models. If the setting cannot be read, none are sent.
  • Raw table rows and columns with many distinct values are not sent.
  • The model is a Model Serving endpoint in your workspace, from an allow-list your admin controls.
  • Output must match a strict schema, and AI output is labelled in the app.
The 100-row dry-run preview grid with the note that the source is untouched. Numbered markers label the parts.
Dry-run · sample data. Markers: 1 the 100-row preview, nothing written, 2 check rows against your request, 3 proceed to run.
Known limits. A table over 10,000 rows is profiled on a 500-row sample, and the profiler picks checks from column names, so it can miss a defect in an unusually named column. Rolling back a productionized recipe is recorded but does not yet disable its schedule. Data Cleansing needs the Team tier or above. See the Data Cleansing guide.

See DataPulse AI on your own workspace.

A 30-minute walk-through of Spend, Data Cleansing and Pipelines, then a read-only Free Health Check on one of your workspaces if it fits.

Book a demoSee pricing