🧱 Databricks Data Matching Wizard

1Prerequisites

Before using the Databricks Data Matching Wizard, you will need:

• An Interzoid API Key: Register for an account and obtain your unique API license key. This key authenticates your requests and tracks usage credits.
• A running SQL warehouse and a personal access token: The token must belong to a user or service principal with SELECT on the tables you want to match (and USE CATALOG and USE SCHEMA on their parents). A dedicated service principal with read-only grants is recommended; see Connect to Databricks.
• Network access from Interzoid: Most workspaces accept connections from anywhere. If yours has an IP access list, add Interzoid's addresses to it first; see Network Access.
• Available Credits: Each record processed consumes one API credit. The wizard counts the rows in your table and shows an estimate before you run, so you can confirm your balance covers the job.

2Launch the Wizard and Enter Your API Key

Open the Databricks Data Matching Wizard in your browser. Before beginning, enter your Interzoid API key in the top-right area of the header bar. Your key will be saved in your browser for future sessions.

• API Key Field: Type or paste your API key into the input field in the header. Click the lock/eye icon to toggle visibility. The API Key label links to your account page.
• Check Credits: Click the Credits button to see your current credit balance. The balance is also used for the cost estimate on the Run tab. Approved accounts may run a negative balance: the wizard shows a negative balance in amber with a note, not as an error, and the job proceeds with usage beyond the balance billed to the account.
• Language Selection: Click the language dropdown in the navigation bar to switch between any of the 17 supported languages. You can also set the language via URL parameter: ?lang=fr for French, ?lang=ja for Japanese, and so on.
• Resources & Support: The Resources menu links to this guide, a sample match report, Interzoid Central Command, and support.

Once your API key is entered, click Get Started on the introduction screen to begin the wizard.

3Select a Matching Function

The wizard presents six matching functions. Choose the one that matches your data and use case. Each card shows a description, the credit cost per record, how many input columns the function needs, and the column parameters it requires. The function you used most recently is marked Last used and pre-selected the next time you open the wizard.

Single-Column Functions

These functions analyze one column of data to find matches:

Combination Functions

These functions use two columns together for higher matching precision:

Click on the card for your chosen function (or select it with the keyboard), then click Next to proceed.

4Connect to Databricks

Enter the SQL warehouse details, click Connect, and then use the cascading dropdown menus to navigate to your data. The wizard connects to the warehouse in real time and presents the available objects at each level. The warehouse itself is identified by its HTTP path, so there is no separate warehouse selection step.

Connection Flow

Two Ways to Enter the Connection

The toggle at the top of the form switches between them:

• Connection Fields: Enter the workspace hostname, the warehouse HTTP path, and a personal access token. There is no port or SSL setting: Databricks is always reached over HTTPS on port 443. The catalog is chosen from a dropdown after connecting.
• Connection String: Paste a complete DSN in the Databricks Go driver form: token:dapi...@dbc-a1b2c3d4-e5f6.cloud.databricks.com:443/sql/1.0/warehouses/abc123?catalog=main. The token, workspace, and warehouse path all travel inside it. When the string names a catalog the wizard goes straight to schema selection. A copy button and a show/hide toggle sit next to the field.

Where to Find Each Value

Everything the wizard needs is on two pages of the Databricks workspace UI. The same values work on AWS, Azure, and Google Cloud.

Network Access

The wizard does not run in your browser's network. The matching API connects to your SQL warehouse from Interzoid's servers. Most workspaces accept connections from any address, in which case nothing needs to change. If your workspace has an IP access list enabled, connections from unlisted addresses are refused with a 403 error, so add the following addresses to the allow list (Admin settings, Security, IP access list, or the ip-access-lists API):

Workspaces with Private Link only (no public front end) cannot be reached from outside your network; use a workspace with a public endpoint, or contact support about a dedicated connector.

Recommended Setup: a Read-Only Service Principal

For ongoing use, create a service principal (or a dedicated user) that can read the tables you want to match and nothing else, generate its token, and grant it access to the warehouse. Access can be revoked at any time by revoking the token or the grants. Run these in a SQL editor as a metastore or catalog admin:

-- Unity Catalog grants for the principal that owns the token
GRANT USE CATALOG ON CATALOG main TO `interzoid-reader`;
GRANT USE SCHEMA  ON SCHEMA  main.sales TO `interzoid-reader`;
GRANT SELECT      ON SCHEMA  main.sales TO `interzoid-reader`;   -- every table in the schema

-- Or a single table instead of the whole schema
GRANT SELECT ON TABLE main.sales.customers TO `interzoid-reader`;

-- The principal also needs CAN USE on the SQL warehouse (Permissions button on the warehouse page)

Enter the principal's token in the wizard. SELECT granted at the schema level covers tables created later; USE CATALOG and USE SCHEMA are what make the catalog and schema appear in the dropdowns.

After Connecting

• Connect Button: Click Connect to sign in. If the connection succeeds, the wizard loads your catalogs (or, in connection string mode, the schemas of the catalog named in the string). If it fails, the message says which part went wrong: a hostname that could not be resolved, an invalid or expired token, an HTTP path that does not match a warehouse, an address blocked by an IP access list, or a warehouse that is still starting (wait a minute and try again).
• Catalog: Select the catalog containing the table you want to match against. The wizard then loads the schemas within that catalog.
• Schema: Select the schema that contains your target table. default is pre-selected when it exists.

Once you have selected a catalog and schema, click Next to proceed to table and column selection. To connect to a different warehouse, click Disconnect.

5Select Table, Columns, and Options

Choose the table to match against, confirm which columns hold the data to match on, and decide what the output should contain.

Table Selection

The wizard presents a dropdown of all tables and views in the schema you selected, including external tables, materialized views, and tables shared with you through Delta Sharing. Choosing one does three things at once: it loads the column names, fetches the first five rows as a preview, and counts the rows so the Run tab can show a cost estimate. The row count appears directly under the table dropdown.

Match Columns

For each matching parameter required by your chosen function, the wizard looks at the column names and pre-selects the most likely one (a column named company_name for Company Name, street_address for Address, and so on). Pre-selected columns are marked, and you can change any of them from the dropdown. Under each selection the wizard shows what the first record will actually send to the matching engine, so a wrong guess is obvious before you run.

• Single-column functions: Select one column for the matching parameter.
• Combination functions: Select two different columns, one for each parameter. The two columns must be different.

Table Preview

Below the column selectors, a preview of the first rows of the table highlights the columns you have mapped, so you can see the data in context. Every column type is rendered as text, including struct, array, map, decimal, and timestamp columns. If a preview cannot be loaded for a table, the mapping still works from the column names.

Output Columns

Use the checkboxes to choose which columns from the table appear in the match report. The match columns are always included and show as checked and locked, so you only need to tick the additional columns you want. Select all and Select none buttons are available for wide tables.

Output Options

These settings are remembered in your browser across sessions.

• Include header row in output: When enabled (default), the first line of the output lists the column names, with simkey as the last name when similarity keys are shown. The spreadsheet view uses this row to label its columns.
• Show similarity keys in output: When enabled (default), each output record includes the generated similarity key as the last column. Records with the same key are matches. Disable this for clean output with only the selected data columns (the spreadsheet view can then no longer group matched records).
• Show matching records only: When enabled (default), only records that have at least one other matching record are returned, grouped into clusters. Disable it to see every record in the table after processing.
• Keep original record order: Available only when all records are returned. When enabled, output row 1 is table row 1 and so on, which is what you want if you plan to load the similarity keys back alongside the original data. When disabled, similar records are sorted together by similarity key.

Click Next when your selections and options are configured.

6Review and Run

The final screen shows a summary of all your selections: matching function, workspace, HTTP path, catalog, schema, table, column assignments, and output options. Below the summary are the number of records to process and the estimated cost in credits, with a banner comparing the estimate to your current balance:

• Green: your balance covers the run.
• Amber: your balance is below the estimate.
• Blue: your balance is negative. Approved accounts may run a negative balance; the job proceeds and usage beyond the balance is billed to the account.
• Grey: the balance has not been checked yet; click Check Credits.

Click the green Run Match button to start processing. The wizard will:

• Validate your API key and check your account's credit position for the job.
• Connect to the SQL warehouse and read the selected columns from your table.
• Process each record through the selected matching algorithm using concurrent workers for performance.
• Generate the match report with records sorted and grouped into clusters of matching entries.

A progress indicator is shown while the job runs. Processing time depends on the number of records: most tables complete within seconds, while very large tables (up to 500,000 records) may take a minute or more. If a run fails, the reason (a rejected job size, a credit limit, a dropped connection) is shown in a red banner above the results area.

7Review the Results

When the run completes, a summary line reports how many records came back and, when similarity keys are included, how many match groups they form. The results panel below it shows the first 1,000 rows of the output; for larger results a note says how many rows there are in total, and the Spreadsheet View, Open as Text File, and Save Results buttons always work on the complete output.

Spreadsheet View

Click Spreadsheet View to open the results in a grid designed for reviewing match reports:

• Grouping: matched records are tinted in alternating shades per similarity group and boxed, so each cluster reads as a unit. When all records are returned, records with no match partner stay plain.
• Search: filters every row by any cell value.
• Sort: click a column header to sort ascending, again for descending, and a third time to restore the original order. Sorting by the similarity key column regroups the records.
• Columns: drag a header edge to resize, double-click it to fit the content, or use the fit-all button. The key column is highlighted.
• Paging and full screen: large results are shown 500 rows per page (search and sort still cover every row), and the grid can expand to fill the window. Press Escape to close it.

Example Output

For a company name match on a table with columns id, company, address, city, state, with similarity keys and the header row enabled:

id,company,address,city,state,simkey
1041,IBM Corporation,1 New Orchard Rd,Armonk,NY,d477E1d7sG6dja3hDNsk9P
2288,I.B.M. Corp,1 New Orchard Road,Armonk,NY,d477E1d7sG6dja3hDNsk9P

1107,Microsoft Inc.,1 Microsoft Way,Redmond,WA,k8Rp2mNx4wQjL9vB3cYh7T
1932,Microsoft Corporation,One Microsoft Way,Redmond,WA,k8Rp2mNx4wQjL9vB3cYh7T
3015,MSFT Corp,1 Microsoft Way,Redmond,WA,k8Rp2mNx4wQjL9vB3cYh7T

In this example, the first cluster contains two records identified as variations of IBM, and the second cluster contains three records identified as variations of Microsoft. The last column in each row is the similarity key; all records sharing the same key are considered matches.

Loading the Keys Back into Databricks

With the primary key and the similarity key in the output, consolidation becomes ordinary SQL. Upload the saved file with the Catalog Explorer "Create or modify table" uploader, or stage it in a volume and use COPY INTO, then join on the primary key:

CREATE TABLE main.sales.company_simkeys (id INT, simkey STRING);

COPY INTO main.sales.company_simkeys
FROM '/Volumes/main/sales/uploads/company-name-only_databricks_matches.csv'
FILEFORMAT = CSV
FORMAT_OPTIONS ('header' = 'true', 'inferSchema' = 'true');

-- Every record with at least one match partner, with its group
SELECT c.*, k.simkey,
       COUNT(*) OVER (PARTITION BY k.simkey) AS group_size
FROM   main.sales.customers c
JOIN   main.sales.company_simkeys k ON k.id = c.id
ORDER  BY k.simkey, c.id;

Run the match with Show matching records only turned off and Keep original record order on if you want a key for every row rather than only the matched ones.

8Save Your Results

Click Save Results above the results panel (or in the spreadsheet view) to download the complete match report as a CSV file. On supported browsers, a save dialog lets you choose the file name and location; on other browsers, the file downloads automatically. Open as Text File opens the complete output in a new browser tab instead.

The saved file is clean, delimited text that can be imported directly into spreadsheets, databases, or other data processing tools for further analysis.

9API Access for Developers

The Databricks Data Matching Wizard is powered by a REST API that can also be called directly from your own applications, notebooks, or Databricks Workflows. The /match and /discover endpoints accept the same parameters either as a GET query string or, recommended, as a JSON body in a POST request. Use POST so the access token never appears in a URL or a server log.

Example: Match (POST)

$ curl -X POST "https://match.interzoid.com/match" \
       -H "Content-Type: application/json" \
       -d '{
    "driver": "databricks",
    "db_host": "dbc-a1b2c3d4-e5f6.cloud.databricks.com",
    "db_http_path": "/sql/1.0/warehouses/abc123def456",
    "db_token": "dapiXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "db_database": "main",
    "db_schema": "sales",
    "table": "customers",
    "columns": "id,company,address,city,state",
    "function": "company-name-only",
    "company_column_name": "company",
    "apikey": "YOUR_API_KEY",
    "showkeys": "true",
    "matchesonly": "true",
    "includeheaders": "true"
  }'

Example: Match with a Connection String (GET)

A complete DSN can be passed in db_host, in which case db_token, db_http_path and (when present as ?catalog=) db_database are taken from the string. It must be URL-encoded in a query string:

$ curl "https://match.interzoid.com/match?\
driver=databricks&\
db_host=token%3AdapiXXXXXXXX%40dbc-a1b2c3d4-e5f6.cloud.databricks.com%3A443%2Fsql%2F1.0%2Fwarehouses%2Fabc123def456%3Fcatalog%3Dmain&\
db_schema=sales&\
table=customers&\
columns=id,company,address,city,state&\
function=company-name-only&\
company_column_name=company&\
apikey=YOUR_API_KEY&\
showkeys=true&\
matchesonly=true&\
includeheaders=true"

Databricks Match API Parameters

Catalog, schema, table, and column names are backtick-quoted by the API. Databricks stores them in lower case and matches them case-insensitively, so pass them as /discover returns them.

Discovery API

The wizard's dropdown menus, table preview, and row count are powered by a /discover endpoint that you can also call directly. It takes the same connection parameters as /match plus a resource. The warehouse is fixed by the HTTP path, so resource=warehouses returns an empty list and a shared client can skip ahead; resource=databases lists Unity Catalog catalogs:

# List catalogs (also validates the token and warehouse). Shown as GET for brevity;
# POST a JSON body with the same fields to keep the token out of the URL.
$ curl "https://match.interzoid.com/discover?driver=databricks&db_host=dbc-a1b2c3d4-e5f6.cloud.databricks.com&db_http_path=/sql/1.0/warehouses/abc123def456&db_token=dapiXXXX&resource=databases"

# List schemas within a catalog
$ curl "https://match.interzoid.com/discover?...&db_database=main&resource=schemas"

# List tables and views within a schema
$ curl "https://match.interzoid.com/discover?...&db_database=main&db_schema=sales&resource=tables"

# List columns within a table
$ curl "https://match.interzoid.com/discover?...&db_database=main&db_schema=sales&table=customers&resource=columns"

# Count the rows in a table (for a cost estimate; served from Delta statistics)
$ curl "https://match.interzoid.com/discover?...&db_database=main&db_schema=sales&table=customers&resource=rowcount"

# First five rows of a table, as column names plus rows
$ curl "https://match.interzoid.com/discover?...&db_database=main&db_schema=sales&table=customers&resource=preview"

The list resources and rowcount return a JSON object with the names (or the count) in items; preview adds a rows array:

{"resource":"databases","items":["main","analytics","hive_metastore"]}

{"resource":"rowcount","items":["12480"]}

{"resource":"preview","items":["id","company","city"],"rows":[["1041","IBM Corporation","Armonk"],["2288","I.B.M. Corp","Armonk"]]}

The /match API returns plain text output identical to what the wizard displays, making it suitable for piping into other tools or storing directly as a file.