🧱 Databricks Data Matching Wizard

Connect directly to your Databricks lakehouse and find matching clusters, no file exports needed

The Databricks Data Matching Wizard is a browser-based, step-by-step tool that connects directly to your Databricks SQL warehouse and uses Interzoid's AI-powered similarity algorithms to identify and cluster matching records within your tables. Whether you need to deduplicate company names, match individual names, or find address variations, the wizard walks you through the entire process, from connecting to Databricks through selecting tables and columns to generating a downloadable match report.

The wizard supports 17 languages and guides you through a cascading connection flow: enter your connection details, then select from your available catalogs, schemas, and tables using dropdown menus populated directly from your workspace. No need to manually look up or type object names.

The wizard works with Databricks workspaces on AWS, Azure, and Google Cloud, reading through a SQL warehouse. Unity Catalog and legacy hive_metastore workspaces are both supported. Because Databricks is reached over HTTPS on port 443, no special outbound firewall rule is needed for the connection.

1Prerequisites

Before using the Databricks Data Matching Wizard, you will need:

An Interzoid API Key: Register for an account and obtain your unique API license key. This key authenticates your requests and tracks usage credits.
A Databricks SQL Warehouse: Any running or startable SQL warehouse in your workspace. Its hostname and HTTP path are both shown on the warehouse's Connection details tab.
A Personal Access Token: Created in your workspace under Settings then Developer then Access tokens. The token is used in place of a username and password.
Read Permissions: The token inherits the permissions of the user who created it. That user needs USE CATALOG and USE SCHEMA on the objects you want to browse, plus SELECT on the table you want to match, and CAN USE on the SQL warehouse itself.
Available Credits: Each record processed consumes one API credit. Ensure your account has sufficient credits for the number of records in your table.

2Launch the Wizard and Enter Your API Key

Open the Databricks Data Matching Wizard in your browser. Before beginning, enter your Interzoid API key in the top-right area of the header bar. Your key will be saved in your browser for future sessions.

API Key Field: Type or paste your API key into the input field in the header. Click the lock/eye icon to toggle visibility.
Check Credits: Click the Credits button to verify your current credit balance before starting a job.
Language Selection: Click the language dropdown in the navigation bar to switch between any of the 17 supported languages. The entire wizard interface will update immediately. You can also set the language via URL parameter: ?lang=fr for French, ?lang=ja for Japanese, etc.

Once your API key is entered, click Get Started on the introduction screen to begin the wizard.

Launch the Databricks Data Matching Wizard

3Select a Matching Function

The wizard presents six matching functions. Choose the one that matches your data and use case. Each function card shows a description and the column parameters it requires.

Single-Column Functions

These functions analyze one column of data to find matches:

Function	Use Case	Column Required
Company Name Matching	Match variations like "IBM", "I.B.M. Corp", "International Business Machines"	Company Name
Individual Name Matching	Match "James Johnston", "Jim Johnston", "J. Johnston" as the same person	Full Name
Street Address Matching	Match "400 E Broadway St" with "400 East Broadway Street"	Address

Combination Functions

These functions use two columns together for higher matching precision:

Function	Use Case	Columns Required
Company + Address	Higher precision matching using both company name and street address	Company Name, Address
Company + Full Name	Contact deduplication using company and individual name	Company Name, Full Name
Address + Full Name	Person-at-address matching using address and individual name	Address, Full Name

Click on the card for your chosen function, then click Next to proceed.

4Connect to Databricks

Enter your connection details and use the cascading dropdown menus to navigate to your data. The wizard connects to Databricks in real time and presents the available objects at each level.

Databricks does have compute warehouses, but unlike Snowflake there is no warehouse selection step: the SQL warehouse is identified by the HTTP path you supply, so it is chosen before connecting rather than after. The Unity Catalog catalog then takes the place a database occupies on other engines, making the flow credentials, catalog, schema.

Connection Flow

Credentials → Catalog → Schema

The wizard offers two ways to supply your connection details. Use the toggle at the top of the form to switch between them.

Option A: Connection Fields

Three fields, all of which come from your workspace. The first two are on the same screen in the Databricks UI.

Workspace Hostname: The server hostname of your workspace (e.g. dbc-a1b2c3d4-e5f6.cloud.databricks.com), shown on the SQL warehouse's Connection details tab. Enter the hostname alone, without https://.
HTTP Path: Identifies which SQL warehouse to use (e.g. /sql/1.0/warehouses/abc123def456), listed directly beneath the hostname on that same tab.
Access Token: A personal access token, created under Settings then Developer then Access tokens. Tokens begin with dapi and are shown only once when created.

There is no port field and no SSL mode. Databricks is always reached over HTTPS on port 443, so both are fixed and neither needs configuring.

Option B: Connection String

Paste a complete connection string instead of filling in the three fields individually. This is the native driver DSN, which carries the token, workspace, and warehouse path together:

# Native driver form
token:dapi123456789@dbc-a1b2c3d4-e5f6.cloud.databricks.com:443/sql/1.0/warehouses/abc123def456

# The same string with a catalog and schema already selected
token:dapi123456789@dbc-a1b2c3d4-e5f6.cloud.databricks.com:443/sql/1.0/warehouses/abc123def456?catalog=main&schema=sales

The literal word token is the username half of the credential and is always the same; your personal access token follows the colon. When a catalog parameter is present the wizard skips the catalog dropdown and goes straight to schema selection.

The connection string field is masked by default. Use the eye icon to reveal it and the clipboard icon to copy it, which is useful if you want to save the string elsewhere or reuse it in another tool.

Tip: Databricks access tokens contain only characters that are safe in a connection string, so no percent-encoding is needed. If you are storing the string anywhere, remember that it contains a live credential.

Connecting and Selecting Your Data

Connect Button: Click Connect to validate your connection details. If the connection succeeds, the wizard loads your available databases into a dropdown menu. If it fails, an error message will indicate what went wrong.
Catalog: Select the Unity Catalog catalog containing the table you want to match against. The wizard then loads the schemas within it. On a workspace without Unity Catalog this lists hive_metastore alone, which is the correct entry point there.
Schema: Select the schema that contains your target table. If a default schema exists, the wizard preselects it.

Tip: Click Disconnect to reset the connection and enter different details, such as when switching to another database server.

Saving Your Connection Details

Below the credentials is a Remember these connection details on this browser toggle. It is off by default, and nothing connection-related is saved until you turn it on.

When enabled: The wizard stores your connection mode, workspace hostname, HTTP path, access token, and connection string in your browser so you do not have to re-enter them next time.
When disabled: Any previously saved values are erased immediately, not merely left in place.
On shared computers: Because the stored values include your access token, leave this toggle off.

Note: Use a Read-Only Service Principal

The wizard only reads from your lakehouse, and it is good practice to connect with an identity that can do nothing else. A personal access token inherits everything its creator can do, so a dedicated service principal with only read grants is the safer choice:

-- Run in a SQL editor, as a catalog or metastore administrator
GRANT USE CATALOG ON CATALOG main TO `interzoid-reader`;
GRANT USE SCHEMA ON SCHEMA main.sales TO `interzoid-reader`;
GRANT SELECT ON TABLE main.sales.customers TO `interzoid-reader`;

-- Or grant SELECT across the whole schema
GRANT SELECT ON SCHEMA main.sales TO `interzoid-reader`;

The service principal also needs CAN USE on the SQL warehouse, granted from the warehouse's Permissions dialog. Access can be revoked at any time with REVOKE, or by deleting the token from the workspace.

Note: The First Connection May Be Slow

SQL warehouses stop automatically after a period of inactivity to avoid consuming compute. Connecting to a stopped warehouse starts it, and that cold start typically takes several seconds, occasionally longer on a classic warehouse. If your first connection attempt times out, the warehouse was almost certainly starting rather than unreachable, and a second attempt moments later will usually succeed.

To avoid the wait entirely, start the warehouse from the Databricks UI before running the wizard, or use a serverless warehouse, which starts considerably faster than a classic one.

Unity Catalog and hive_metastore: Unity Catalog introduces a three-level namespace of catalog, schema, and table, which is what the wizard's dropdowns follow. Workspaces that predate it use a two-level namespace under a single hive_metastore catalog. Both work: on a legacy workspace the catalog step simply offers hive_metastore, and everything below it behaves the same.

Security Note: Your access token is transmitted over HTTPS with each connection and discovery request and is not retained by Interzoid after the session ends. It is stored in your browser only if you enable the Remember these connection details toggle described above. Because a personal access token grants the same access its creator has, treat it with the same care as a password.

Once you have selected a catalog and schema, click Next to proceed to table and column selection.

5Select Table, Columns, and Options

Choose the table to match against and configure which columns to use for matching and which columns to include in the output.

Table Selection

The wizard presents a dropdown of all tables and views available in the schema you selected. Choose the one containing the records you want to match. Once selected, the wizard loads the column names from that table in their natural column order.

Note on Complex Types: Columns holding ARRAY, MAP, or STRUCT values are read as their JSON text form rather than being rejected, so they can be selected as output columns. They are rarely useful as match columns, though: matching works on the names, addresses, and text that identify an entity, so a plain string column is almost always the right choice.

Match Columns

For each matching parameter required by your chosen function, select the corresponding column from the dropdown. For example, if you chose "Company Name Matching," select the column that contains company names.

Single-column functions: Select one column for the matching parameter.
Combination functions: Select two different columns, one for each parameter. The two columns must be different.

Output Columns

Use the checkboxes to select which columns from the table you want to include in the match report output. At least one column must be selected. The match columns are automatically included even if you don't check them separately, which ensures the data you matched on always appears in the results.

Tip: Select additional identifying columns (like id, city, state, etc.) to make the match report more useful for downstream analysis. You don't need to include every column, just the ones that help you identify and act on the matches.

Output Options

Show Similarity Keys: When enabled (default), each output record includes the generated similarity key as the last column. Records with the same key are matches. Disable this if you want clean output with only the selected data columns.
Matches Only: When enabled (default), only records that have at least one other matching record are shown. Disable this to see every record in the table after processing, sorted by similarity key.

Click Next when your selections and options are configured.

6Review and Run

The final screen shows a summary of all your selections: matching function, workspace, HTTP path, catalog, schema, table, column assignments, and output options. Review these carefully before proceeding.

Click the green Run Match button to start processing. The wizard will:

Validate your API key and check that your account has sufficient credits for the job.
Connect to Databricks and read the selected columns from your table.
Process each record through the selected matching algorithm using concurrent workers for performance.
Generate the match report with records sorted and grouped into clusters of matching entries.

A progress indicator is shown while the job runs. Processing time depends on the number of records, and on whether the SQL warehouse was already running. Most tables complete within seconds once the warehouse is warm, while very large tables may take a minute or more.

Note: If the matching engine encounters too many errors, the job will stop early and display an error message. Verify that the SQL warehouse is still running and that the selected table and columns are accessible.

7Interpret the Results

The match report appears in the results panel at the bottom of the screen. Records are organized into clusters, groups of records that the AI has determined to be matches. Each cluster is separated by a blank line for readability.

Example Output

For a company name match on a Databricks table with columns company, address, city, state and similarity keys enabled:

IBM Corporation,1 New Orchard Rd,Armonk,NY,d477E1d7sG6dja3hDNsk9P
I.B.M. Corp,1 New Orchard Road,Armonk,NY,d477E1d7sG6dja3hDNsk9P

Microsoft Inc.,1 Microsoft Way,Redmond,WA,k8Rp2mNx4wQjL9vB3cYh7T
Microsoft Corporation,One Microsoft Way,Redmond,WA,k8Rp2mNx4wQjL9vB3cYh7T
MSFT Corp,1 Microsoft Way,Redmond,WA,k8Rp2mNx4wQjL9vB3cYh7T

In this example, the first cluster contains two records identified as variations of IBM, and the second cluster contains three records identified as variations of Microsoft. The last column in each row is the similarity key. All records sharing the same key are considered matches.

8Save Your Results

Click the Save Results button above the results panel to download the match report as a CSV file. On supported browsers, a save dialog will appear allowing you to choose the file name and location. On other browsers, the file will download automatically.

The saved file is clean, delimited text that can be imported directly into spreadsheets, databases, or other data processing tools for further analysis.

Data Pipeline Integration: The match report output has no metadata or headers, just data rows and blank-line cluster separators. This makes it suitable for direct use in automated data pipelines.

View All Interzoid APIs Getting Started Guide CSV/TSV Matching Wizard Docs

The Databricks Data Matching Wizard makes it easy to discover hidden duplicates and matching records directly within your Databricks lakehouse. It delivers clean, actionable match reports that help you improve the quality, consistency, and value of your data assets, without ever exporting a file. If you have any questions or need assistance, don't hesitate to reach out to our support team.