EDITION: ENTERPRISE

Flyway's advanced capabilities for Databricks is in public preview, it lets you compare and deploy schema changes for Databricks, using the same state and migrations workflows you already use for other database types. It connects to Databricks over JDBC and works with Lakebase and Lakehouse Unity Catalog.

Advanced capabilities include

• Understanding changes made to a development schema
• Scripting those changes out so each object can be versioned controlled giving you an object-level history to see who, changed what, when and why 
• Optionally generating versioned and undo migrations scripts, which account for dependencies
• Reporting on changes as part of a CI/CD pipeline so you can understand how many objects will be touched at a glance and drill into more details with a side-by-side diff of the changes
• Check for drift to ensure environments reflect what is in version control to catch issues sooner before releasing to production 

Supported objects

The Preview supports the most common object types:

• Schemas, including their comment, owner, default collation, properties (DBPROPERTIES), tags and predictive-optimization setting
• Tables, including partitioning and table rebuilds where a change can't be made in place
• Columns, including data types, NOT NULL, DEFAULT values, and GENERATED / identity columns
• Views
• Primary keys, foreign keys and check constraints
• Comments at the schema, view and column level

Risky changes are flagged with a warning before you deploy. Review these carefully:

• Adding a NOT NULL column without a default.
• Rebuilding a table that contains an identity column - the IDs are regenerated, so the resulting values may differ from the originals.

Limitations

This Preview is not feature-complete. It is intended to work for common workflows; you may hit gaps outside them. Known limitations include:

• Lakebase or Lakehouse Unity Catalog only. The legacy hive_metastore catalog is not supported.
• Object coverage is limited to the types listed above. Other object types are not yet supported - including stored procedures, functions, materialized views, streaming tables, volumes, foreign tables and shallow clones.
• Unsupported properties of a supported object are not scripted. Where an object we support has a feature we don't yet model - for example a row filter or column mask on a table - that feature is ignored rather than captured or deployed.

We expect to expand coverage based on the feedback we get during the Preview.  Email us with your feedback and what you'd like us to work on next.

Turning the Preview off

Advanced Databricks support is on by default. To turn it off:

• Flyway Desktop: go to Settings > Preview features and turn the Databricks option off.
• Flyway CLI: set the environment variable RGCOMPARE_DATABRICKS to false on the machine running the CLI.

Giving feedback

We'd love to hear how you get on with the Preview. Tell us what works, what doesn't, and what you'd like to see next.

Please complete our quick survey or email us with your feedback.