# Databricks

{% hint style="info" %}
The following documentation is for **Databricks v1.1.0**.
{% endhint %}

## Overview

Databricks is a unified analytics platform for data engineering, data science, and machine learning. This integration enables automated data pipeline management, notebook execution, and analytics workflows within your Databricks workspace.

## Setup

The following integrations need to be connected to your Kognitos workspace:

* **Databricks**

### Steps

Follow these steps to connect the integration in Kognitos:

{% stepper %}
{% step %}
**Navigate**

Using the left navigation menu, go to **Integrations** → **Explore Integrations**.
{% endstep %}

{% step %}
**Find**

Search for the integration and click on it.
{% endstep %}

{% step %}
**Connect**

Click on <kbd>**Connect**</kbd> to add a connection to the integration.
{% endstep %}

{% step %}
**Configure**

Add a name for the connection. You'll be prompted for [**authentication**](#authentication) details if needed. Then, click on <kbd>**Connect**</kbd>.
{% endstep %}
{% endstepper %}

## Authentication

Use one of the following authentication methods to connect this integration in Kognitos. Each method has its own configuration requirements.

### Connect using Cluster URL and Access Token

Connects to Databricks using the provided access token.

| Label        | Description                                | Type        |
| ------------ | ------------------------------------------ | ----------- |
| Cluster URL  | The Databricks cluster URL                 | `text`      |
| Access Token | The access token to be used for connecting | `sensitive` |

## Actions

The following actions are available in the **Databricks** integration:

### 1. Append a file to a table

Append data from a file to an existing table.

### 2. Append a table to a databricks table

Append data from a Kognitos table to an existing Databricks table.

### 3. Create a databricks table from a table

Create a table in Databricks from a Kognitos table.

### 4. Create databricks table from a file

Create a table from an uploaded file.

### 5. Get a catalog

Get a catalog by its name.

### 6. Get file at a path

Get file metadata for a file at the given path.

### 7. Get some catalog's schema

Get a schema from a catalog by its name.

### 8. Get some warehouses

List all warehouses in the workspace.

### 9. Upload a file to databricks

Upload a file to a catalog volume.

## Concepts

### Databricks warehouse

Warehouse in Databricks.

| Field Name | Description                            | Type   |
| ---------- | -------------------------------------- | ------ |
| `id`       | The unique identifier of the warehouse | `text` |
| `name`     | The name of the warehouse              | `text` |

### Databricks file

File in Databricks.

| Field Name    | Description                                          | Type   |
| ------------- | ---------------------------------------------------- | ------ |
| `schema`      | The schema containing the file                       | `json` |
| `path`        | The path to the file                                 | `text` |
| `file_type`   | The MIME type of the file                            | `text` |
| `file_format` | The file format extension (e.g., csv, json, parquet) | `text` |

### Databricks schema

Schema in Databricks.

| Field Name | Description                        | Type   |
| ---------- | ---------------------------------- | ------ |
| `name`     | The name of the schema             | `text` |
| `catalog`  | The catalog containing this schema | `json` |

### Databricks catalog

Catalog in Databricks.

| Field Name | Description             | Type   |
| ---------- | ----------------------- | ------ |
| `name`     | The name of the catalog | `text` |


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.kognitos.com/guides/platform/integrations/databricks.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.