Databricks SDK for JavaScript


> [!WARNING]
> ## ⚠️ PREVIEW - NOT FOR PRODUCTION USE
>
> This SDK is in preview and is subject to change without notice.
>
> - ❌ Do NOT use in production environments
> - ⚠️ Breaking changes may occur at any time
> - 🔬 APIs are experimental and unstable
> - 📝 Use for development and testing only
>
> For production use cases, please wait for the stable 1.0.0 release.

The Databricks SDK for JavaScript provides a convenient way to interact with Databricks REST APIs in Node.js and TypeScript applications. It covers all public Databricks REST API operations and the SDK's internal HTTP client is robust and handles failures on different levels by performing intelligent retries.

Table of Contents

- Installation
- Getting Started
- Authentication
- Default Authentication Flow
- Databricks Native Authentication
- Azure Native Authentication
- Google Cloud Platform Native Authentication
- Configuration File
- Environment Variables
- Usage Examples
- Listing Clusters
- Creating and Waiting for Clusters
- Submitting Jobs
- Unity Catalog Operations
- Long-Running Operations
- Pagination
- Error Handling
- Configuration
- Timeouts
- HTTP Proxy
- Rate Limiting
- Logging and Debugging
- Requirements
- Testing
- Contributing
- License

Installation

Install the SDK using your preferred package manager:

$3

``bash
npm install @databricks/sdk-experimental
`

$3

`bash
pnpm add @databricks/sdk-experimental
`

Getting Started

The simplest way to get started is to create a WorkspaceClient and let it automatically detect your credentials:

`typescript
import {WorkspaceClient} from "@databricks/sdk-experimental";

// Create client (automatically detects credentials)
const client = new WorkspaceClient();

// List all clusters
for await (const cluster of client.clusters.list({})) {
console.log(cluster.cluster_name);
}

// Get current user
const user = await client.currentUser.me();
console.log(Logged in as: ${user.user_name});
`

Authentication

If you use Databricks configuration profiles or Databricks-specific environment variables for Databricks authentication, the only code required to start working with a Databricks workspace is:

`typescript
import {WorkspaceClient} from "@databricks/sdk-experimental";
const client = new WorkspaceClient();
`

$3

The SDK tries the following authentication methods in order until it succeeds:

1. Databricks Native Authentication (PAT tokens, OAuth M2M, Workload Identity Federation)
2. Azure Native Authentication (Azure CLI, Azure MSI, Azure Client Secret)
3. Google Cloud Platform Native Authentication (GCP credentials, default application credentials)
4. If unsuccessful, returns an authentication error

For each authentication method, the SDK searches for compatible credentials in the following locations:

1. Credentials explicitly passed to the Config constructor
2. Databricks-specific environment variables (e.g., DATABRICKS_HOST, DATABRICKS_TOKEN)
3. Configuration file (~/.databrickscfg)
4. Cloud provider authentication (Azure CLI, GCP default credentials, AWS instance profiles)

$3

#### Personal Access Token (PAT)

The most common authentication method for development:

`typescript
import {Config, WorkspaceClient} from "@databricks/sdk-experimental";

const config = new Config({
host: "https://your-workspace.cloud.databricks.com",
token: "dapi...",
});
const client = new WorkspaceClient(config);
`

Or using environment variables:

`bash
export DATABRICKS_HOST="https://your-workspace.cloud.databricks.com"
export DATABRICKS_TOKEN="dapi..."
`

`typescript
import {WorkspaceClient} from "@databricks/sdk-experimental";

// Automatically uses environment variables
const client = new WorkspaceClient();
`

#### OAuth Machine-to-Machine (M2M) - Service Principal

For production service-to-service authentication:

`typescript
import {Config, WorkspaceClient} from "@databricks/sdk-experimental";

const config = new Config({
host: "https://your-workspace.cloud.databricks.com",
clientId: "your-client-id",
clientSecret: "your-client-secret",
});
const client = new WorkspaceClient(config);
`

Environment variables:

`bash
export DATABRICKS_HOST="https://your-workspace.cloud.databricks.com"
export DATABRICKS_CLIENT_ID="your-client-id"
export DATABRICKS_CLIENT_SECRET="your-client-secret"
`

#### OAuth User-to-Machine (U2M) - External Browser

For interactive user authentication with SSO:

`typescript
import {Config, WorkspaceClient} from "@databricks/sdk-experimental";

const config = new Config({
host: "https://your-workspace.cloud.databricks.com",
clientId: "your-client-id",
authType: "oauth-u2m",
});
const client = new WorkspaceClient(config);
`

The SDK will automatically open a browser for user authentication.

$3

#### Azure CLI

If you've authenticated with az login, the SDK will automatically use those credentials:

`typescript
import {Config, WorkspaceClient} from "@databricks/sdk-experimental";

const config = new Config({
host: "https://adb-..azuredatabricks.net",
authType: "azure-cli",
});
const client = new WorkspaceClient(config);
`

#### Azure Managed Service Identity (MSI)

For applications running in Azure:

`typescript
import {Config, WorkspaceClient} from "@databricks/sdk-experimental";

const config = new Config({
host: "https://adb-..azuredatabricks.net",
azureClientId: "your-managed-identity-client-id",
});
const client = new WorkspaceClient(config);
`

#### Azure Service Principal

`typescript
import {Config, WorkspaceClient} from "@databricks/sdk-experimental";

const config = new Config({
host: "https://adb-..azuredatabricks.net",
azureClientId: "your-client-id",
azureClientSecret: "your-client-secret",
azureTenantId: "your-tenant-id",
});
const client = new WorkspaceClient(config);
`

$3

#### GCP Default Application Credentials

If you've authenticated with gcloud auth application-default login:

`typescript
import {Config, WorkspaceClient} from "@databricks/sdk-experimental";

const config = new Config({
host: "https://your-workspace.gcp.databricks.com",
authType: "google-credentials",
});
const client = new WorkspaceClient(config);
`

#### GCP Service Account

`typescript
import {Config, WorkspaceClient} from "@databricks/sdk-experimental";

const config = new Config({
host: "https://your-workspace.gcp.databricks.com",
googleServiceAccount: "service-account@project.iam.gserviceaccount.com",
});
const client = new WorkspaceClient(config);
`

$3

Create a ~/.databrickscfg file:

`ini
[DEFAULT]
host = https://your-workspace.cloud.databricks.com
token = dapi...

[PROD]
host = https://prod-workspace.cloud.databricks.com
token = dapi...

[STAGING]
host = https://staging-workspace.cloud.databricks.com
token = dapi...
`

Use profiles in your code:

`typescript
import {Config, WorkspaceClient} from "@databricks/sdk-experimental";

// Use DEFAULT profile
const client = new WorkspaceClient();

// Use specific profile
const prodClient = new WorkspaceClient(new Config({profile: "PROD"}));
const stagingClient = new WorkspaceClient(new Config({profile: "STAGING"}));
`

$3

The SDK supports the following environment variables:

| Variable | Description |
| ----------------------------------- | ---------------------------------------------------- |
| DATABRICKS_HOST | Workspace or account URL |
| DATABRICKS_TOKEN | Personal access token |
| DATABRICKS_CLIENT_ID | OAuth client ID / Azure client ID |
| DATABRICKS_CLIENT_SECRET | OAuth client secret / Azure client secret |
| DATABRICKS_ACCOUNT_ID | Databricks account ID (for account-level operations) |
| DATABRICKS_AZURE_TENANT_ID | Azure tenant ID |
| DATABRICKS_GOOGLE_SERVICE_ACCOUNT | GCP service account email |
| DATABRICKS_AUTH_TYPE | Force specific auth type |

Usage Examples

$3

`typescript
import {WorkspaceClient} from "@databricks/sdk-experimental";

const client = new WorkspaceClient();

// List all clusters
for await (const cluster of client.clusters.list({})) {
console.log(Cluster: ${cluster.cluster_name} (${cluster.cluster_id}));
console.log( State: ${cluster.state});
console.log( Spark Version: ${cluster.spark_version});
}
`

$3

`typescript
import {WorkspaceClient} from "@databricks/sdk-experimental";

const client = new WorkspaceClient();

// Create cluster and wait for it to be running
const waiter = await client.clusters.create({
cluster_name: "my-cluster",
spark_version: "13.3.x-scala2.12",
node_type_id: "i3.xlarge",
num_workers: 2,
autotermination_minutes: 60,
});

// Wait with progress callback
const cluster = await waiter.wait({
timeout: 15 60 1000, // 15 minutes
onProgress: (c) => {
console.log(Cluster state: ${c.state});
},
});

console.log(Cluster ${cluster.cluster_id} is running!);
`

$3

`typescript
import {WorkspaceClient} from "@databricks/sdk-experimental";

const client = new WorkspaceClient();

// Submit a one-time job run
const waiter = await client.jobs.submit({
run_name: "my-job-run",
tasks: [
{
task_key: "main",
notebook_task: {
notebook_path: "/Users/user@example.com/my-notebook",
source: "WORKSPACE",
},
new_cluster: {
spark_version: "13.3.x-scala2.12",
node_type_id: "i3.xlarge",
num_workers: 2,
},
},
],
});

// Wait for job to complete
const run = await waiter.wait({
timeout: 30 60 1000, // 30 minutes
onProgress: (r) => {
console.log(Job state: ${r.state?.life_cycle_state});
},
});

console.log(Job completed with result state: ${run.state?.result_state});
`

$3

`typescript
import {WorkspaceClient} from "@databricks/sdk-experimental";

const client = new WorkspaceClient();

// List all catalogs
console.log("Catalogs:");
for await (const catalog of client.catalogs.list({})) {
console.log( - ${catalog.name});
}

// Create a new schema
const schema = await client.schemas.create({
name: "my_schema",
catalog_name: "my_catalog",
comment: "Schema for analytics",
});
console.log(Created schema: ${schema.full_name});

// List tables in a schema
console.log("Tables in my_catalog.my_schema:");
for await (const table of client.tables.list({
catalog_name: "my_catalog",
schema_name: "my_schema",
})) {
console.log( - ${table.full_name});
}

// Create a volume
const volume = await client.volumes.create({
catalog_name: "my_catalog",
schema_name: "my_schema",
name: "my_volume",
volume_type: "MANAGED",
});
console.log(Created volume: ${volume.full_name});
`

Workspace Examples

The SDK includes 18+ complete, runnable examples covering common Databricks workspace operations. These examples demonstrate best practices and patterns for real-world use cases.

$3

- Clusters (3 examples) - Create, list, start/stop clusters
- Jobs (4 examples) - Submit, create, trigger, and monitor jobs
- Unity Catalog (4 examples) - Manage catalogs, schemas, tables, volumes, and permissions
- SQL Warehouses (2 examples) - Create warehouses and execute queries
- ML/MLflow (3 examples) - Track experiments, log models, model registry
- Notebooks (2 examples) - Upload and export notebooks
- Files/DBFS (2 examples) - Upload and download files

Each example is self-contained, includes cleanup code, and demonstrates SDK best practices like the waiter pattern, pagination, and error handling.

See Workspace Examples for the complete catalog with usage instructions.

$3

`bash


Run any example with npx ts-node

npx ts-node examples/workspace/clusters/create-cluster.ts

Or make executable and run directly

chmod +x examples/workspace/clusters/create-cluster.ts
./examples/workspace/clusters/create-cluster.ts
`

$3

`typescript
import {WorkspaceClient, Time, TimeUnits} from "@databricks/sdk-experimental";

const client = new WorkspaceClient({});

// Submit a one-time job run
const waiter = await client.jobs.submit({
run_name: my-job-${Date.now()},
tasks: [
{
task_key: "main_task",
notebook_task: {
notebook_path: "/Shared/my-notebook",
source: "WORKSPACE",
},
new_cluster: {
spark_version: "13.3.x-scala2.12",
node_type_id: "i3.xlarge",
num_workers: 2,
},
},
],
});

// Wait for completion with progress updates
const result = await waiter.wait({
timeout: new Time(20, TimeUnits.minutes),
onProgress: async (run) => {
console.log(State: ${run.state?.life_cycle_state});
},
});

console.log(
Job ${result.state?.result_state === "SUCCESS" ? "succeeded" : "failed"}
);
`

For authentication examples and setup, see Authentication Examples.

Long-Running Operations

Many Databricks operations are asynchronous. The SDK provides a Waiter interface for these operations:

`typescript
import {WorkspaceClient} from "@databricks/sdk-experimental";

const client = new WorkspaceClient();

// Start an async operation
const waiter = await client.clusters.create({
cluster_name: "my-cluster",
spark_version: "13.3.x-scala2.12",
node_type_id: "i3.xlarge",
num_workers: 2,
});

// Configure wait behavior
const cluster = await waiter.wait({
timeout: 20 60 1000, // 20 minutes
onProgress: (intermediate) => {
console.log(Current state: ${intermediate.state});
console.log(State message: ${intermediate.state_message});
},
});

console.log(Operation completed! Cluster ID: ${cluster.cluster_id});
`

$3

The SDK automatically retries transient errors (rate limits, temporary unavailability):

`typescript
import {WorkspaceClient} from "@databricks/sdk-experimental";
import {Config} from "@databricks/sdk-experimental";

const config = new Config({
host: "https://your-workspace.cloud.databricks.com",
token: "dapi...",
retryTimeoutSeconds: 600, // Retry for up to 10 minutes
});

const client = new WorkspaceClient(config);
`

Pagination

The SDK automatically handles pagination using async iterators:

`typescript
import {WorkspaceClient} from "@databricks/sdk-experimental";

const client = new WorkspaceClient();

// Automatically handles pagination
for await (const job of client.jobs.list({limit: 25})) {
console.log(Job: ${job.settings?.name} (${job.job_id}));
}

// Collect all results into an array
const allJobs = [];
for await (const job of client.jobs.list({})) {
allJobs.push(job);
}
console.log(Total jobs: ${allJobs.length});
`

Error Handling

The SDK provides structured error handling:

`typescript
import {WorkspaceClient} from "@databricks/sdk-experimental";
import {ApiError} from "@databricks/sdk-experimental";

const client = new WorkspaceClient();

try {
const cluster = await client.clusters.get({cluster_id: "invalid-id"});
} catch (error) {
if (error instanceof ApiError) {
console.error(API Error: ${error.message});
console.error(Status Code: ${error.statusCode});
console.error(Error Code: ${error.errorCode});

// Check for specific error types
if (error.statusCode === 404) {
console.error("Cluster not found");
} else if (error.statusCode === 403) {
console.error("Permission denied");
} else if (error.statusCode === 429) {
console.error("Rate limited - will automatically retry");
}
} else {
console.error("Unexpected error:", error);
}
}
`

The SDK automatically retries the following errors:

- 429: Too Many Requests (rate limiting)
- 503: Service Temporarily Unavailable
- Connection errors: Network issues, timeouts

Configuration

$3

Configure HTTP and retry timeouts:

`typescript
import {Config, WorkspaceClient} from "@databricks/sdk-experimental";

const config = new Config({
host: "https://your-workspace.cloud.databricks.com",
token: "dapi...",
httpTimeoutSeconds: 60, // HTTP request timeout (default: 5s)
retryTimeoutSeconds: 300, // Total retry timeout (default: 300s)
});

const client = new WorkspaceClient(config);
`

$3

Configure an HTTP proxy for all requests:

`typescript
import {Config, WorkspaceClient} from "@databricks/sdk-experimental";

const config = new Config({
host: "https://your-workspace.cloud.databricks.com",
token: "dapi...",
proxy: "http://proxy.example.com:8080",
});

const client = new WorkspaceClient(config);
`

Or using environment variables:

`bash
export HTTPS_PROXY="http://proxy.example.com:8080"
export HTTP_PROXY="http://proxy.example.com:8080"
`

$3

Configure client-side rate limiting:

`typescript
import {Config, WorkspaceClient} from "@databricks/sdk-experimental";

const config = new Config({
host: "https://your-workspace.cloud.databricks.com",
token: "dapi...",
rateLimitPerSecond: 10, // Maximum 10 requests per second
});

const client = new WorkspaceClient(config);
`

Logging and Debugging

Enable debug logging to troubleshoot issues:

`typescript
import {setLogger, WorkspaceClient} from "@databricks/sdk-experimental";

// Set up custom logger
setLogger({
log: (level, message, context) => {
console.log([${level}] ${message}, context);
},
});

const client = new WorkspaceClient();
`

Use environment variables for debugging:

`bash


Enable header debugging (redacts sensitive data)

export DATABRICKS_DEBUG_HEADERS=true

Limit debug output truncation

export DATABRICKS_DEBUG_TRUNCATE_BYTES=2000
`

Requirements

- Node.js: 18.0 or higher
- TypeScript: 4.5 or higher (if using TypeScript)

Authentication Examples

The SDK supports 10+ authentication methods. See Authentication Examples for complete, runnable examples covering:

- Personal Access Tokens (PAT)
- OAuth Machine-to-Machine (Service Principals)
- OAuth User-to-Machine (SSO)
- Configuration file profiles
- Azure authentication (CLI, MSI, Service Principal)
- Google Cloud Platform authentication
- AWS instance profiles
- Custom credentials providers

Each example includes setup instructions and can be run with npx ts-node.

Authentication Examples

The SDK supports 10+ authentication methods. See Authentication Examples for complete, runnable examples covering:

- Personal Access Tokens (PAT)
- OAuth Machine-to-Machine (Service Principals)
- OAuth User-to-Machine (SSO)
- Configuration file profiles
- Azure authentication (CLI, MSI, Service Principal)
- Google Cloud Platform authentication
- AWS instance profiles
- Custom credentials providers

Each example includes setup instructions and can be run with npx ts-node.

Testing

$3

Run unit tests:

`bash
npm run test
`

$3

Most tests are integration tests and must be run against a live cluster. Configure the following environment variables:

| Name | Value |
| ------------------------- | --------------------------------------------------------------------------------------------------- |
| DATABRICKS_HOST | Hostname of the Databricks workspace (starts with https://) |
| DATABRICKS_TOKEN | Personal access token |
| TEST_DEFAULT_CLUSTER_ID | (optional) ID of a cluster to run tests against. If missing, tests will create a cluster on demand. |

Run integration tests:

`bash
export DATABRICKS_HOST="https://your-workspace.cloud.databricks.com"
export DATABRICKS_TOKEN="dapi..."
npm run test
``

Contributing

See CONTRIBUTING.md for development setup and contribution guidelines.

The Databricks SDK for JavaScript is developed as part of the broader Databricks SDK ecosystem. We welcome contributions from the community!

License

Apache License 2.0. See LICENSE for details.