This tutorial shows you how to connect Flyway to Databricks Lakebase, Databricks' managed PostgreSQL service, using browser sign-in, a service principal or a native Postgres password. For Databricks Lakehouse (Unity Catalog), see Tutorial - Connect to a Databricks instance instead.

Prerequisites:

• A Databricks workspace with a Lakebase project. The examples use the project's default production branch and primary compute.
• A database to deploy to, for example databricks_postgres, and, if you'll generate migrations, a second database for your shadow environment.
• Flyway Desktop 9.10.6 or later, if you're connecting from Flyway Desktop.
• To generate migrations, view changes or check for drift: Flyway Teams or Enterprise in Flyway Desktop, or Flyway Enterprise for the generate and check -drift commands.

Flyway connects to Lakebase with the PostgreSQL JDBC driver, which ships with Flyway, so there's nothing extra to install.

Getting your connection details

1. In your Databricks workspace, open "Lakebase" and select your project.
2. Click "Connect" and choose the "Java (JDBC)" connection string.
3. Note the host, for example ep-my-endpoint.database.uksouth.azuredatabricks.net, and the database name.
4. Note your workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net, and the project ID. Browser sign-in and service principals need both.

Flyway connects with a JDBC URL like this:

jdbc:postgresql://<host>/<database>?sslmode=require

sslmode=require is required, because Lakebase only accepts encrypted connections. See the Databricks Lakebase driver reference for more detail.

Lakebase uses Postgres user name and password authentication. The password is either a short-lived Databricks OAuth token, issued for the Postgres role of a Databricks identity, or a native Postgres role password.

It is recommended to not write secrets directly into flyway.toml if you intend to check it into version control. The examples below read them from environment variables, but you can also use any of Flyway's secret resolvers.

Connecting with browser sign-in

OAuth user-to-machine (U2M) opens a browser so you sign in with your own Databricks identity. Interactive use only.

You need a Postgres role for your Databricks identity, named after your email address. Lakebase creates one for the project owner. If you don't own the project, ask its owner to add one for you: in the Lakebase app, open the branch overview and click "Add role", then choose your Databricks identity.

[environments.development]
url = "jdbc:postgresql://ep-my-endpoint.database.uksouth.azuredatabricks.net/databricks_postgres?sslmode=require"
user = "${lakebase.user}"
password = "${lakebase.token}"

[environments.development.resolvers.lakebase]
workspaceUrl = "https://adb-1234567890123456.7.azuredatabricks.net"
project = "my-project"

Connecting with a service principal

OAuth machine-to-machine (M2M) is the best fit for pipelines. Your pipeline signs in as a Databricks service principal, so deployments don't depend on a person's credentials.

The Lakebase Resolver doesn't support service principals, so your pipeline fetches a database credential before running Flyway and passes it in as the password.

Setting up the service principal

1. Add a service principal to your Databricks account, assign it to your workspace and turn on its "Workspace access" entitlement.
2. Click your username, then "Settings" > "Identity and access". Next to "Service principals", click "Manage", select your service principal, and open the "Secrets" tab.
3. Click "Generate secret" and set its lifetime, up to 730 days.
4. Copy the secret and the client ID. Databricks shows the secret only once.
5. In the Lakebase SQL editor, create a Postgres role for the service principal. The role name is its client ID:

CREATE EXTENSION IF NOT EXISTS databricks_auth;
SELECT databricks_create_role('<client-id>', 'SERVICE_PRINCIPAL');

Connecting using the service principal

Add a step to your pipeline that uses the Databricks CLI to generate a database credential. With DATABRICKS_HOST , DATABRICKS_CLIENT_ID  and DATABRICKS_CLIENT_SECRET  set, the CLI signs in as the service principal. The endpoint is projects/<project-id>/branches/<branch-id>/endpoints/<compute-id> :

export LAKEBASE_TOKEN=$(databricks postgres generate-database-credential \
  projects/my-project/branches/production/endpoints/primary --output json | jq -r .token)

Then add an environment to flyway.toml. Set user to the service principal's client ID and password to the credential:

[environments.production]
url = "jdbc:postgresql://ep-my-endpoint.database.uksouth.azuredatabricks.net/databricks_postgres?sslmode=require"
user = "${env.DATABRICKS_CLIENT_ID}"
password = "${env.LAKEBASE_TOKEN}"

The Flyway Desktop connection dialog doesn't offer service principal sign-in, so add this environment to flyway.toml directly.

Connecting with a Postgres password

A native Postgres role with a password works without any Databricks sign-in. Databricks recommends OAuth over passwords where you can.

Password connections are turned off by default for new Lakebase projects. Turn them on first: see Manage password connections.

Creating the role

1. In the Lakebase app, open the branch overview and click "Add role".
2. Choose a Postgres password role, enter a name, and copy the password. Lakebase shows it only once.

Connecting using the password

[environments.development]
url = "jdbc:postgresql://ep-my-endpoint.database.uksouth.azuredatabricks.net/databricks_postgres?sslmode=require"
user = "flyway_deployer"
password = "${env.LAKEBASE_PASSWORD}"

Granting permissions

On the Databricks side, a user or service principal needs permission to use the Lakebase project. Can use.

In Postgres, the role Flyway connects as needs to connect to the database and to use and create objects in each schema Flyway works with. Replace <role> with the email address, the service principal's client ID or the password role's name:

GRANT CONNECT ON DATABASE databricks_postgres TO "<role>";
GRANT USAGE, CREATE ON SCHEMA public TO "<role>"; -- repeat for each schema Flyway works with

To change or drop objects created by another role, Flyway's role needs to own them or be a member of the role that does. The simplest option for a deployment role is to make it the owner of the schemas it manages. Your migration scripts may need further privileges. See Database permissions.

Testing the connection

From the command line, run:

flyway testConnection -environment=production

This opens a connection to Databricks without changing anything. With Flyway Enterprise, it also checks that the comparison engine can connect. In Flyway Desktop, use "Test connection" in the connection dialog.

Reference

• Databricks Lakebase driver reference
• Lakebase Resolver
• testConnection
• Databricks: Connect an external app to Lakebase using the API