Databricks Advanced MCP Server

An advanced Model Context Protocol (MCP) server that gives AI assistants deep visibility into your Databricks workspace — 43 tools covering dependency scanning, impact analysis, notebook review, job/pipeline operations, SQL execution, catalog management, compute & warehouse control, and Unity Catalog volumes.

Features

Demo

Demo teaser

<details>

<summary>Click to play full video</summary>

https://github.com/user-attachments/assets/579282ca-bb26-4244-b0c6-3ad26050aca3

</details>

Covers SQL execution, dependency scanning, impact analysis, notebook review, and job/pipeline operations.

Quick Start

Prerequisites

• Python 3.11+
• [uv](https://docs.astral.sh/uv/) — fast Python package manager
• A Databricks workspace with a SQL warehouse
• A Databricks personal access token

Other auth methods: The Databricks SDK supports unified authentication — if you don't set DATABRICKS_TOKEN, it will fall back to Azure CLI, managed identity, or .databrickscfg. The .env setup below uses a PAT for simplicity.

>

Don't have a Databricks workspace yet? See infra/INSTALL.md for a one-command Azure deployment using Bicep.

1. Install

#### Option A: Install from PyPI (recommended)

uv pip install databricks-advanced-mcp

Or with pip:

pip install databricks-advanced-mcp

#### Option B: Install from source

git clone https://github.com/henrybravo/databricks-advanced-mcp-server.git
cd databricks-advanced-mcp-server

Create and activate a virtual environment:

Windows (PowerShell)

uv venv .venv
.\.venv\Scripts\Activate.ps1
uv pip install -e .

macOS / Linux

uv venv .venv
source .venv/bin/activate
uv pip install -e .

2. Configure

cp .env.example .env

Edit .env with your Databricks credentials:

# Azure Databricks:
DATABRICKS_HOST=https://adb-xxxx.azuredatabricks.net
# Databricks on AWS / GCP:
# DATABRICKS_HOST=https://dbc-xxxx.cloud.databricks.com

DATABRICKS_TOKEN=dapi_your_token
DATABRICKS_WAREHOUSE_ID=your_warehouse_id

# Optional (defaults shown)
# Azure workspaces typically use "main"; AWS/GCP workspaces use "workspace"
DATABRICKS_CATALOG=main
DATABRICKS_SCHEMA=default

3. Add to your IDE

Create .vscode/mcp.json in your project to register the MCP server with VS Code / GitHub Copilot.

#### Option A: PyPI install (recommended)

If you installed from PyPI (pip install databricks-advanced-mcp), the databricks-mcp CLI is available on your PATH:

{
  "servers": {
    "databricks-mcp": {
      "type": "stdio",
      "command": "databricks-mcp",
      "env": {
        "DATABRICKS_HOST": "https://adb-xxxx.azuredatabricks.net",
        "DATABRICKS_TOKEN": "dapi_your_token",
        "DATABRICKS_WAREHOUSE_ID": "your_warehouse_id"
      }
    }
  }
}

#### Option B: Virtual environment (source install)

If you cloned the repo and installed into a local .venv, point directly to the Python interpreter:

Windows

{
  "servers": {
    "databricks-mcp": {
      "type": "stdio",
      "command": "${workspaceFolder}/.venv/Scripts/python.exe",
      "args": ["-m", "databricks_advanced_mcp.server"],
      "envFile": "${workspaceFolder}/.env"
    }
  }
}

macOS / Linux

{
  "servers": {
    "databricks-mcp": {
      "type": "stdio",
      "command": "${workspaceFolder}/.venv/bin/python",
      "args": ["-m", "databricks_advanced_mcp.server"],
      "envFile": "${workspaceFolder}/.env"
    }
  }
}

#### Multiple Workspaces

Each MCP server instance connects to exactly one Databricks workspace. To work with multiple workspaces simultaneously, register a separate server entry per workspace — each with its own credentials:

{
  "servers": {
    // AWS / GCP workspace
    "databricks-cloud": {
      "type": "stdio",
      "command": "databricks-mcp",
      "env": {
        "DATABRICKS_HOST": "https://dbc-xxxx.cloud.databricks.com",
        "DATABRICKS_TOKEN": "dapi_cloud_token",
        "DATABRICKS_WAREHOUSE_ID": "cloud_warehouse_id",
        "DATABRICKS_CATALOG": "workspace"
      }
    },
    // Azure workspace
    "databricks-azure": {
      "type": "stdio",
      "command": "databricks-mcp",
      "env": {
        "DATABRICKS_HOST": "https://adb-xxxx.azuredatabricks.net",
        "DATABRICKS_TOKEN": "dapi_azure_token",
        "DATABRICKS_WAREHOUSE_ID": "azure_warehouse_id",
        "DATABRICKS_CATALOG": "main"
      }
    }
  }
}

Alternatively, with a source install you can use separate .env files per workspace:

{
  "servers": {
    "databricks-cloud": {
      "type": "stdio",
      "command": "${workspaceFolder}/.venv/bin/python",
      "args": ["-m", "databricks_advanced_mcp.server"],
      "envFile": "${workspaceFolder}/.env"
    },
    "databricks-azure": {
      "type": "stdio",
      "command": "${workspaceFolder}/.venv/bin/python",
      "args": ["-m", "databricks_advanced_mcp.server"],
      "envFile": "${workspaceFolder}/.env_azure"
    }
  }
}

4. Start using

Once configured, your AI assistant can call any of the 43 tools below. Here are example prompts organized by domain:

Explore your data

• "What tables exist in the `analytics` schema?"
• "Show me the schema and metadata for `main.sales.orders`"
• "Run a query that counts and sums orders by status from `main.sales.orders`"

Unity Catalog & schemas

• "List all catalogs I have access to"
• "What schemas exist in the analytics catalog?"
• "Describe the `main.default` schema"
• "Create a new schema called `staging` in the analytics catalog"

Understand dependencies

• "Build the full workspace dependency graph"
• "What are the upstream and downstream dependencies of `main.default.customers`?"
• "Scan the `/Shared/mandated_broker_v2_etl_pipeline` notebook for table references"
• "Scan all jobs and show their table dependencies"

Assess impact before making changes

• "What would break if I drop the `customer_id` column from `main.default.customers`?"
• "What's the impact of removing the `amount` column and renaming `status` to `order_status` in `main.sales.orders`?"

Review notebook quality

• "Review `/Shared/mandated_broker_v2_etl_pipeline` for performance issues"
• "Review `/Shared/analysis` for all issues — performance, coding standards, and optimizations"

Monitor jobs and pipelines

• "List all jobs in the workspace"
• "What's the current status of job 12345?"
• "Show me the pipeline status for my DLT pipeline"
• "Trigger a new run of job 67890 with parameter env=prod"

Compute & SQL warehouses

• "Show me the status of cluster abc-123"
• "List all running clusters"
• "Stop the dev SQL warehouse"
• "What warehouses are currently active?"

Workspace & volumes

• "Export the ETL notebook as source"
• "What's the status of the notebook at /Workspace/Users/me/analysis?"
• "What files are in the raw-data volume?"
• "Read the config.json file from the settings volume"

MCP Tools

SQL & Tables (3 tools)

Dependency Scanning (4 tools)

Graph Operations (3 tools)

Impact Analysis & Review (2 tools)

Job & Pipeline Ops (6 tools)

Catalog & Schema (5 tools)

Compute (5 tools)

SQL Warehouses (4 tools)

Workspace Ops (7 tools)

UC Volumes (4 tools)

Configuration Reference

Security note: The execute_query tool can run any SQL against your warehouse, including DDL (DROP TABLE, ALTER TABLE) and DML (DELETE, UPDATE). Use a least-privilege service principal (see Security & Governance below) rather than a personal admin PAT in production environments.

Cloud Provider Notes

This server is tested against Azure Databricks and Databricks on AWS (.cloud.databricks.com). Key differences:

All tools work on both platforms. Set DATABRICKS_CATALOG to match your workspace's default catalog.

Security & Governance

Recommended: Service Principal with Least Privilege

Avoid storing a personal admin PAT in .env or VS Code config. Instead, create a dedicated service principal with only the permissions required:

-- Grant read access on the catalog and schema
GRANT USE CATALOG ON CATALOG main TO `sp-databricks-mcp`;
GRANT USE SCHEMA ON SCHEMA main.default TO `sp-databricks-mcp`;
GRANT SELECT ON SCHEMA main.default TO `sp-databricks-mcp`;

-- For job operations (trigger_rerun, trigger_job_run, get_job_status)
-- Grant CAN_MANAGE_RUN on specific jobs only, via the Databricks UI or API

Set DATABRICKS_TOKEN to the service principal's OAuth token or M2M secret. The Databricks SDK supports OAuth M2M authentication natively — no PAT required.

Tool Risk Levels

Unity Catalog ACLs

All execute_query and get_table_info calls respect Unity Catalog row/column-level security. If the token's principal lacks SELECT on a table, the operation fails with a permission error — expected and correct behaviour.

Query Guard

To limit execute_query to read-only statements, restrict the SQL warehouse's channel policy or use a dedicated read-only warehouse for this MCP server.

Infrastructure (Optional)

If you need to provision a new Azure Databricks workspace, the infra/ directory contains:

• `main.bicep` — Azure Bicep template (Premium SKU, Unity Catalog enabled)
• `deploy.ps1` — One-command PowerShell deployment script
• `INSTALL.md` — Detailed step-by-step deployment guide

cd infra
./deploy.ps1 -ResourceGroupName rg-databricks-mcp -Location eastus2

Development

# Install with dev dependencies
uv pip install -e ".[dev]"

# Run tests (excluding live integration tests)
uv run pytest tests/ --ignore=tests/test_workspace_ops_live.py -v

# Run tests with coverage report
uv run pytest tests/ --cov=src/databricks_advanced_mcp --cov-report=term-missing --ignore=tests/test_workspace_ops_live.py

# Lint
uv run ruff check src/ tests/

# Type check
uv run mypy src/

Architecture

src/databricks_advanced_mcp/
├── server.py              # FastMCP server + CLI entry point
├── config.py              # Pydantic settings from env vars
├── client.py              # Databricks SDK client factory
├── tools/                 # MCP tool implementations (43 tools across 13 modules)
│   ├── __init__.py        # Central registration of all tool modules
│   ├── sql_executor.py    # SQL execution (1 tool)
│   ├── table_info.py      # Table metadata (2 tools)
│   ├── dependency_scanner.py # Scan notebooks/jobs/pipelines (4 tools)
│   ├── graph_ops.py       # Build/query/refresh dependency graph (3 tools)
│   ├── impact_analysis.py # Impact analysis (1 tool)
│   ├── notebook_reviewer.py # Notebook review (1 tool)
│   ├── job_pipeline_ops.py  # Job & pipeline operations (6 tools)
│   ├── workspace_listing.py # Workspace listing (1 tool)
│   ├── workspace_ops.py   # Workspace mutations + read/delete (6 tools)
│   ├── catalog_ops.py     # Unity Catalog & schema management (5 tools)
│   ├── compute_ops.py     # Cluster management (5 tools)
│   ├── warehouse_ops.py   # SQL warehouse management (4 tools)
│   └── volume_ops.py      # Unity Catalog volumes (4 tools)
├── parsers/               # Code parsing engines
│   ├── sql_parser.py      # sqlglot-based SQL extraction
│   ├── notebook_parser.py # Databricks notebook cell parsing
│   └── dlt_parser.py      # DLT pipeline definition parsing
├── graph/                 # Dependency graph
│   ├── models.py          # Node, Edge, DependencyGraph data models
│   ├── builder.py         # Graph builder (orchestrates scans)
│   └── cache.py           # In-memory graph cache with TTL
└── reviewers/             # Notebook review rule engines
    ├── performance.py     # Performance anti-patterns
    ├── standards.py       # Coding standards checks
    └── suggestions.py     # Optimization suggestions

License

MIT

Contributions welcome! See CONTRIBUTING.md for setup instructions, PR checklist, and a list of wanted features.