Redgate Flyway

EDITION: ENTERPRISE

Flyway's advanced capabilities for Databricks is in public preview, it lets you compare and deploy schema changes for Databricks, using the same state and migrations workflows you already use for other database types. It connects to Databricks over JDBC and works with Lakebase and Lakehouse Unity Catalog.

This feature is in Preview. It is not guaranteed to be released, and is not officially supported, but we welcome your feedback on bugs and improvements. Databricks support is enabled by default and can be turned off (see Turning the Preview off below). If you have questions or feedback about this feature, please get in touch with our Development Team.

Advanced capabilities include

Understanding changes made to a development schema
Scripting those changes out so each object can be versioned controlled giving you an object-level history to see who, changed what, when and why
Optionally generating versioned and undo migrations scripts, which account for dependencies
Reporting on changes as part of a CI/CD pipeline so you can understand how many objects will be touched at a glance and drill into more details with a side-by-side diff of the changes
Check for drift to ensure environments reflect what is in version control to catch issues sooner before releasing to production

Supported objects

Lakehouse Lakebase

Create a new project with the Databricks type.

The Preview supports the most common object types:

Schemas, including their comment, owner, default collation, properties (DBPROPERTIES), tags and predictive-optimization setting
Tables, including partitioning and table rebuilds where a change can't be made in place
Columns, including data types, NOT NULL, DEFAULT values, and GENERATED / identity columns
Views
Primary keys, foreign keys and check constraints
Comments at the schema, view and column level

Risky changes are flagged with a warning before you deploy. Review these carefully:

Adding a NOT NULL column without a default.
Rebuilding a table that contains an identity column - the IDs are regenerated, so the resulting values may differ from the originals.

Limitations

This Preview is not feature-complete. It is intended to work for common workflows; you may hit gaps outside them. Known limitations include:

Lakehouse Unity Catalog only. The legacy hive_metastore catalog is not supported.
Object coverage is limited to the types listed above. Other object types are not yet supported - including stored procedures, functions, materialized views, streaming tables, volumes, foreign tables and shallow clones.
Unsupported properties of a supported object are not scripted. Where an object we support has a feature we don't yet model - for example a row filter or column mask on a table - that feature is ignored rather than captured or deployed.

Connecting

Copy the JDBC URL from Databricks and paste it into the URL in the connection dialog.. It should look
databricks://<host>:<port>/default;transportMode=http;ssl=1;AuthMech=3;httpPath=/sql/1.0/warehouses/<id>
You'll need to add a semicolon and your catalog to the end of the JDBC URL:
;ConCatalog=<catalog_name>
Include your username and enter the token for the password.

Create a new project with the PostgreSQL type.

The Preview aligns with the advanced capabilities we support for the PostgreSQL database.

We expect to expand coverage based on the feedback we get during the Preview. Email us with your feedback or complete our quick survey.

Turning the Preview off

Advanced Databricks support is on by default. To turn it off:

Flyway Desktop: go to Settings > Preview features and turn the Databricks option off.

Giving feedback

We'd love to hear how you get on with the Preview. Tell us what works, what doesn't, and what you'd like to see next.

Please complete our quick survey or email us with your feedback.