This guideline is deprecated, meaning it is not being maintained with the latest features and guidelines. It is still however, defining a valid ADO pipeline that can be used as a basis for a pipeline. Although tweaks may be required to complete the setup with newer versions of the Flyway CLI.

Please follow this video series for the latest and most up-to-date pipeline setup guidance: https://www.red-gate.com/hub/university/courses/flyway/flyway-desktop-enterprise-implementation/preparing-for-the-poc/proof-of-concept

A working example of how-to setup a flyway pipeline within Azure DevOps, using the declarative YAML pipeline-as-code method.

Section 1 - Introduction

To make the most of Flyway, and the migrations model it offers, it is often desirable to implement Flyway within a CI/CD pipeline. However, with such a wide variety of CI/CD tools out there, it can be difficult to know where to start.

This article has therefore been created to work through two commonly asked questions:

• How do I run Flyway commands inside a pipeline?
• How can I setup a starter Azure DevOps pipeline, using Flyway?

Section 2 - Redgate Documentation

Before we get started, it’s valuable to note that a lot of the content discussed in this document can also be found on various Redgate Documentation websites. See below for helpful resources:

• https://flywaydb.org/documentation/getstarted/
• https://www.red-gate.com/hub/university/courses/flyway

Therefore, if you require any additional knowledge around the topics discussed in this article, it is strongly advised to review the helpful guides provided above.

Section 3 - Getting Started

During this article, the below example YAML pipeline will be referenced. However, pipeline updates can also be found on this github.

name: ADO-self-hosted-pipeline

# This is the default pipeline for a self-hosted Windows agent on Azure Devops.

# Install flyway cli on agent, add flyway to PATH: https://download.red-gate.com/maven/release/org/flywaydb/enterprise/flyway-commandline
# Install python3 on agent and add pip to PATH if staticCodeAnalysis is set to true
# Make sure this file is in the same directory as the migrations folder of the Flyway Enterprise project. 
# Provision a dev, shadow, build databases, as well as any target environments that need to be created: https://documentation.red-gate.com/fd/proof-of-concept-checklist-152109292.html
# Further instructions if needed here: https://documentation.red-gate.com/fd/self-hosted-windows-agent-yaml-pipeline-in-azure-devops-158564470.html
# For video reference, see: https://www.red-gate.com/hub/university/courses/flyway/flyway-desktop/setting-up-a-flyway-desktop-project/basic-flyway-desktop-project-setup-and-configuration

trigger:
  branches:
    include:
      - release
  paths:
    include:
      - migrations/*

parameters:
  
  # IMPORTANT: DO NOT ADD DEPLOYMENT STEPS TO THE BUILD STAGE - THE BUILD IS A DESTRUCTIVE ACTION
  - name: buildStage
    type: object
    default: 

      stage: 'Build'
      displayName: 'Deploy Build'
      executeBuild: true
      variableGroupName: 'build_credentials_variable_group' #userName, password, target_database_JDBC, databaseName

  # This is the extensible definition of your target environments. 
  # Every parameter in deploymentStages corresponds to an environment - here it's Test and Prod.
  # Pay attention to the 'dependsOn' field - this determines order of operations.
  # IMPORTANT: check_JDBC will have schema dropped
  - name: deploymentStages
    type: object
    default: 

    - stage: 'Test'
      dependsOn: 'Build'
      displayName: 'Deploy Test'
      pauseForCodeReview: false
      generateDriftAndChangeReport: true                     #requires check database to be provisioned
      failReleaseIfDriftDetected: false                      #only works if generateDriftAndChangeReport is set to true
      staticCodeAnalysis: false                              #requires python3 installed on agent and pip on PATH
      variableGroupName: 'test_credentials_variable_group'   #userName, password, target_database_JDBC, databaseName, check_JDBC 
    
    - stage: 'Prod'
      dependsOn: 'Test'
      displayName: 'Deploy Prod'
      pauseForCodeReview: true
      generateDriftAndChangeReport: true
	  failReleaseIfDriftDetected: false                      #only works if generateDriftAndChangeReport is set to true                   
      staticCodeAnalysis: false                             
      variableGroupName: 'prod_credentials_variable_group' 

variables:

  # This is the relative path to the migrations folder in your project, such as:
  # $(System.DefaultWorkingDirectory)\project
  # The default is to have the migrations folder in the same directory as the yml file
  WORKING_DIRECTORY: $(System.DefaultWorkingDirectory)

  FLYWAY: 'flyway -user="$(userName)" -password="$(password)" -baselineOnMigrate=true-licenseKey=$(FLYWAY_LICENSE_KEY) -configFiles="$(WORKING_DIRECTORY)\flyway.conf" -locations="filesystem:$(WORKING_DIRECTORY)\migrations"'
  
  BUILD_NAME: 'Build'
  RELEASE_PREVIEW: 'Release-Preview.sql'
  DRIFT_AND_CHANGE_REPORT: 'Drift-And-Change-Report.html'
  DRIFT_AND_CHANGE_REPORT_DISPLAY_NAME: 'Drift And Change Report'
  
  # Place following in flyway_vars variable group
  # FLYWAY_LICENSE_KEY
  # BASELINE_VERSION - make match the baseline version in your project
  # FIRST_UNDO_SCRIPT match the first undo version in your project
  # AGENT_POOL is the agent pool your pipeline will use - default to default
  group: rg_flyway_vars

stages:
  - stage: Build
    pool: $(AGENT_POOL)
    displayName: ${{parameters.buildStage.displayName}} 
    jobs:
    - job: Build
      variables:
      - group: ${{parameters.buildStage.variableGroupName}}
      - group: rg_flyway_vars
      steps:
        
        - script: '$(FLYWAY) clean info -url="$(target_database_JDBC)"'
          failOnStderr: true
          displayName: 'Clean Build DB'
          condition: eq('${{ parameters.buildStage.executeBuild }}', 'true')
          env:
            FLYWAY_CLEAN_DISABLED: false

        - script: '$(FLYWAY) migrate info -url="$(target_database_JDBC)" -baselineVersion=$(BASELINE_VERSION)'
          failOnStderr: true
          condition: eq('${{ parameters.buildStage.executeBuild }}', 'true')
          displayName: 'Validate Migrate Scripts'

        - script: '$(FLYWAY) undo info -url="$(target_database_JDBC)" -target="$(FIRST_UNDO_SCRIPT)"?'
          continueOnError: true
          condition: eq('${{ parameters.buildStage.executeBuild }}', 'true')
          displayName: 'Validate Undo Scripts'

        - task: CopyFiles@2
          inputs:
            targetFolder: '$(System.ArtifactsDirectory)' 

        - task: PublishBuildArtifacts@1
          displayName: 'Publish Build Artifact'
          inputs:
            ArtifactName: '$(BUILD_NAME)'
            PathtoPublish: '$(System.ArtifactsDirectory)'

  - ${{each stage in parameters.deploymentStages}}:
    
    - stage: ${{stage.stage}} 
      pool: $(AGENT_POOL)
      displayName: ${{stage.displayName}} 
      dependsOn: ${{stage.dependsOn}} 
      jobs:
      - job: PreRelease
        displayName: Release Preview
        variables:
        - group: ${{stage.variableGroupName}}
        - group: rg_flyway_vars
        steps:
          - task: DownloadBuildArtifacts@0
            inputs:
              buildType: 'current'
              downloadType: 'single'
              artifactName: '$(BUILD_NAME)'
              downloadPath: '$(System.ArtifactsDirectory)'
      
      - ${{if eq(stage.staticCodeAnalysis, true)}}:
        - job: ChangeReport
          dependsOn: 'PreRelease'
          displayName: Change Report With Code Analysis
          variables:
          - group: ${{stage.variableGroupName}}
          - group: flyway_vars
          steps:
            - script: 'pip install sqlfluff==1.3.2'
              displayName: 'Install SQL Fluff'
              failOnStderr: true
            
            - script: '$(FLYWAY) check -dryrun -changes -drift -code -check.failOnDrift="${{stage.failReleaseIfDriftDetected}}" -check.buildUrl="$(check_JDBC)" -url="$(target_database_JDBC)" -reportFilename="$(System.ArtifactsDirectory)\$(databaseName)-$(Build.BuildId)-$(DRIFT_AND_CHANGE_REPORT)"'
              workingDirectory: '$(WORKING_DIRECTORY)'
              failOnStderr: true
              displayName: '$(DRIFT_AND_CHANGE_REPORT_DISPLAY_NAME)'
              env:
                FLYWAY_CLEAN_DISABLED: false
            
            - task: PublishBuildArtifacts@1
              displayName: 'Publish $(DRIFT_AND_CHANGE_REPORT_DISPLAY_NAME)'
			  condition: always()
              inputs:
                ArtifactName: '$(DRIFT_AND_CHANGE_REPORT_DISPLAY_NAME)'
                PathtoPublish: '$(System.ArtifactsDirectory)\$(databaseName)-$(Build.BuildId)-$(DRIFT_AND_CHANGE_REPORT)'

      - ${{if and(eq( stage.generateDriftAndChangeReport, true), eq( stage.staticCodeAnalysis, false))}}:
        - job: ChangeReport
          displayName: Change Report
          dependsOn: 'PreRelease'
          variables:
          - group: ${{stage.variableGroupName}}
          - group: rg_flyway_vars
          steps:
            - script: '$(FLYWAY) check -dryrun -changes -drift -check.failOnDrift="${{stage.failReleaseIfDriftDetected}}" -check.buildUrl="$(check_JDBC)" -check.buildUser="$(check_password)" -check.buildPassword="$(check_password) -url="$(target_database_JDBC)" -reportFilename="$(System.ArtifactsDirectory)\$(databaseName)-$(Build.BuildId)-$(DRIFT_AND_CHANGE_REPORT)"'
              workingDirectory: '$(WORKING_DIRECTORY)'
              failOnStderr: true
              displayName: '$(DRIFT_AND_CHANGE_REPORT_DISPLAY_NAME)'
              env:
                FLYWAY_CLEAN_DISABLED: false
            
            - task: PublishBuildArtifacts@1
              displayName: 'Publish $(DRIFT_AND_CHANGE_REPORT_DISPLAY_NAME)'
			  condition: always()
              inputs:
                ArtifactName: '$(DRIFT_AND_CHANGE_REPORT_DISPLAY_NAME)'
                PathtoPublish: '$(System.ArtifactsDirectory)\$(databaseName)-$(Build.BuildId)-$(DRIFT_AND_CHANGE_REPORT)'
      
      - ${{if and(eq( stage.generateDriftAndChangeReport, false), eq( stage.staticCodeAnalysis, false))}}:
        - job: ChangeReport
          displayName: Change Report
          dependsOn: 'PreRelease'
          variables:
          - group: ${{stage.variableGroupName}}
          - group: rg_flyway_vars
          steps:
            - script: '$(FLYWAY) check -dryrun -url="$(target_database_JDBC)" -reportFilename="$(System.ArtifactsDirectory)\$(databaseName)-$(Build.BuildId)-$(DRIFT_AND_CHANGE_REPORT)"'
              workingDirectory: '$(WORKING_DIRECTORY)'
              failOnStderr: true
              displayName: '$(DRIFT_AND_CHANGE_REPORT_DISPLAY_NAME)'
              env:
                FLYWAY_CLEAN_DISABLED: false
            
            - task: PublishBuildArtifacts@1
              displayName: 'Publish $(DRIFT_AND_CHANGE_REPORT_DISPLAY_NAME)'
              inputs:
                ArtifactName: '$(DRIFT_AND_CHANGE_REPORT_DISPLAY_NAME)'
                PathtoPublish: '$(System.ArtifactsDirectory)\$(databaseName)-$(Build.BuildId)-$(DRIFT_AND_CHANGE_REPORT)'
      
      - ${{if eq(stage.pauseForCodeReview, true)}}:
        - job: CodeReview
          displayName: Code Review
          dependsOn: 'ChangeReport'
          pool: server
          steps:
            - task: ManualValidation@0
              displayName: 'Review Change Report Prior To Release'
              timeoutInMinutes: 4320 # job times out in 1 hour
              inputs:
                notifyUsers: |
                  user@email.com
                  example@example.com
                instructions: 'Review changes'
      
      - ${{if eq(stage.pauseForCodeReview, false)}}:
        - job: CodeReview
          pool: server
          displayName: Skipping Code Review
          dependsOn: 'ChangeReport'
      
      - job: Deploy
        displayName: Deployment
        dependsOn: 'CodeReview'
        variables:
        - group: ${{stage.variableGroupName}}
        - group: rg_flyway_vars
        steps:

          - script: '$(FLYWAY) info migrate -outOfOrder=true info -url="$(target_database_JDBC)" -baselineVersion=$(BASELINE_VERSION)'
            workingDirectory: $(WORKING_DIRECTORY)
            displayName: ${{stage.displayName}}
            failOnStderr: true
            env:
              FLYWAY_CLEAN_DISABLED: true # clean drops a target DB schema, keep disabled except for build step

Section 4 - Considerations

Before taking a look within the Azure DevOps pipeline, consider as you go the following questions:

• What do I want my automated pipeline to look like?
• How many environments do I want to have automated and will I need any manual intervention gates?
• Do I have/want Linux or Windows based pipeline agents?
• Do I have/want self-hosted or azure-hosted agents?
  • If you're interested in using Azure-Hosted agents, make sure to check out the dockerized equivalent of this article by clicking HERE.

Answers to these, will help guide how you may want to setup your automated processes.

Section 5 - CI/CD Tools

Although this article has a focus on Azure DevOps, it is fundamental to mention that Flyway as a CLI has no pre-requisite for any one individual CI/CD tool. Meaning that depending on your personal pipeline requirements, you are free to select a CI/CD tool that best fits your needs. With the knowledge that the Flyway CLI can be built into any pipeline tool, that has the capability of running a CMD/PowerShell or Bash style script.

Additionally, Flyway has the capability of being installed on a large number of platforms. Making pipelines much more flexible, when it comes to operating system requirements. At the time of writing this article, Flyway installers can be found for the following platforms:

• https://flywaydb.org/download/community
  • This example pipeline uses some Teams only features, therefore make sure to download the Teams Edition CLI (Direct link to the download direction can be found by clicking HERE.)

Section 6 - Azure DevOps

With Azure DevOps being one of the leading SaaS platforms for end-to-end DevOps development and deployment, we often get asked if Flyway is compatible and can be used within its Build and Release pipelines? The answer to this, is a resounding yes! This is because, as mentioned in the previous section, Flyway can be installed on a range of platforms and can be interacted with in a range of fashions. Making it simple to call Flyway using your preferred scripting language, built directly into an Azure DevOps pipeline.

In Section 8, which will be around setting up a basic pipeline, we will be using Azure DevOps’ Declarative pipeline-as-code functionality. This utilizes the concept of a YAML file, which allows users to script out their pipeline in a single markup file.

If you’re unfamiliar with any of this, please find some helpful articles below:

• https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started/pipelines-get-started?view=azure-devops
• https://docs.microsoft.com/en-us/azure/devops/pipelines/create-first-pipeline

Section 7 - Azure DevOps Agents

In order to carry out the pipeline actions, Azure DevOps uses the concept of agents. The most common agents used, are Microsoft-Hosted or Self-Hosted. 

• Microsoft-Hosted
  • These agents are automatically maintained and upgraded by Microsoft. However, each time you run a pipeline, a fresh virtual machine is spun up for each job in the pipeline. Once the job is complete, the virtual machine is then discarded (This means that any changed that a job makes to the virtual machine file system, such as checking out code, will be unavailable to the next job).
• Self-Hosted
  • These agents are setup and managed by yourselves and is used to run jobs within the pipeline. Self-Hosted agents provide more flexibility and control around dependent software that may be required for builds and deployments. Something that is extremely valuable within a Flyway pipeline.
• More information
  • https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops&tabs=browser

For the remainder of this article, I will be presuming Self-Hosted agents are in use, with the following Redgate software installed and functional on the agent machine:

• Flyway CLI
  • https://flywaydb.org/download

Section 7 - Azure DevOps Pipeline Setup

Step 1 - Create the YAML File and Push to Remote Repository

To start the process, it is first necessary to save the YAML pipeline example from Section 3 as a new file in the root directory of your preferred GIT repository. The filename can be anything, for this example I will use the name 'FlywayPipeline.yml. Ensure the new file is then pushed to the remote repo, which in this example will be a repo within Azure DevOps.

Step 2 - Create the new pipeline in Azure DevOps

Whilst in your preferred Azure DevOps organisation, click Pipelines on the left-hand menu. Followed by selecting 'New Pipeline' in the top right-hand corner:

In the following screen, choose the repository location for where the code is coming from. In this example, we are selecting 'Azure Repos GIT', as the source code is housed in the Azure DevOps repos location. However, yours may differ, depending on where the source code is currently housed. Once the code location is selected, simply choose the relevant repository on the following screen.

In the configuration tab, select 'Existing Azure Pipelines YAML file', which will indicate we want to select a YAML file from within our repo. Following this, either enter the location for the file, or select the down arrow to select one from the drop-down list.

Step 3 - Review the YAML Pipeline

Although the example YAML pipeline contains all the relevant steps to get started with Flyway, it is FUNDAMENTAL that the pipeline is reviewed and the variables updated before use. This is due to the highly configurable nature of Flyway and it's ability to connect to the databases and folder locations in your environment.

After reviewing the pipeline file, you'll notice that there are 3 distinct stages. These stages are Build, Test and Prod; designed to mimic a standard customer environment. Within the stages themselves, specifically Test and Prod, the stages have been split further into 'PreRelease' and 'Deployment'. This has been done to help visualize the actions taking place within the pipeline process, in addition to allowing for code review steps to be carried out where requirement (For example, before pushing anything to Prod). With any good pipeline, security should always be a pertinent consideration. Which is why, the pipeline utilizes two types of variables throughout the process. The first being pipeline specific variables, which contain non-sensitive information for use throughout the pipeline. The majority of pipeline specific variables have been recently moved out of the YAML file and into the relevant 'Variable Group'. This has been done to make to make it easier to re-use variables and more easily replicate your pipelines. The second being 'Variable Groups', which provides a secure method of adding pipeline information, without the inherit risk of using plain text values. The setup of both of these are fundamental to the running of the pipeline. Therefore, please follow the instructions below:

Stage Variables

As a means of making it simple to configure the Build/Test and Prod environment details, the variables related to each stage are grouped together in code blocks between lines 86 - 109. As mentioned earlier in the article, all of the environment specific variables have been moved from within the YAML file to the corresponding 'Variable Group' that is referenced in the below stage specific sections. Therefore, it is not necessary to update any of the settings in the below code blocks. Unless additional stages are required, which would entail copying and pasting either the Test or Prod code block. Followed by changing the variables as required, for example updating the stage's name/display name/dependsOn value and variableGroupName.

Although the variables used within each of the three sections are similar, please find a breakdown of what they are used for below:

      stage: 'Build'
      displayName: 'Deploy Build'
      executeBuild: true
      variableGroupName: 'build_credentials_variable_group' #userName, password, target_database_JDBC, databaseName

	- stage: 'Test'
      dependsOn: 'Build'
      displayName: 'Deploy Test'
      pauseForCodeReview: false
      generateDriftAndChangeReport: true                    #requires check database to be provisioned
      failReleaseIfDriftDetected: false
      staticCodeAnalysis: false                              #requires python3 installed on agent and pip on PATH
      variableGroupName: 'test_credentials_variable_group'  #userName, password, target_database_JDBC, databaseName, check_JDBC 

	- stage: 'Prod'
      dependsOn: 'Test'
      displayName: 'Deploy Prod'
      pauseForCodeReview: true
      generateDriftAndChangeReport: true                   #requires check database to be provisioned
      failReleaseIfDriftDetected: false
      staticCodeAnalysis: false                              #requires python3 installed on agent and pip on PATH
      variableGroupName: 'prod_credentials_variable_group'  #userName, password, target_database_JDBC, databaseName, check_JDBC 

• executeBuild (Build Stage Only)
  • This boolean variable can be changed to false, if the build stage should be skipped. The default is true, which means the build stage steps will be executed.
• dependsOn (Test/Prod Stages)
  • Configuring this, informs ADO that there is a dependency for another stage to be successful before starting.
    • I.e - Ensure the Build process has been successful, before attempting Test.
• pauseForCodeReview
  • The ability to review the database code changes before deployment, can be extremely helpful whilst getting comfortable with CI/CD and Flyway. Therefore, setting this to true, will result in a Manual Validation step being present for that stage after the 'PreRelease' steps have completed.
• generateDriftAndChangeReport
  • This variable enables the generation of a single report, created by the Flyway CLI, which details information pertaining to any pending changes that Flyway has for the upcoming deployment. In addition, to a Drift report, which outlines any changes made to the target environment that are not currently known by Flyway and Source Control.
• failReleaseIfDriftDetected
  • This variable informs the Flyway CLI to fail the ADO pipeline, if a drift discrepancy is detected between the target environment and what Flyway believes the target environment should look like (Schema object specific)
• staticCodeAnalysis
  • The Flyway CLI now has the ability to harness code analysis functionality, useful in providing automatic feedback around pending migrations. Warning - This feature requires the installation of Python3 on the agent machine, in addition to PIP being present on the environment variable 'PATH'. If you're unfamiliar with this setup, please leave this variable as false.
• variableGroupName
  • This defines the name of the 'Variable Group' to be utilized within the pipeline stage. Details on how to set these up, can be found in the next section.

Variable Groups

As mentioned earlier in the article, variable groups are used in this pipeline to add an extra level of security around the more sensitive pieces of information. For example, usernames and passwords. 

To setup the variable groups, navigate to the 'Library' tab, found within the 'Pipelines' section of the left hand navigation menu within Azure DevOps. Following this, select the '+ Variable Group' button to create your first group.

For the pipeline to function, it is necessary to create three variable groups, one for each stage. Each group will need to contain the 'userName' and 'password' information that the Flyway CLI will utilize for connecting to that stage's database. If 'integratedSecurity' is enabled, please create the variable name but keep the value empty. In addition to the databaseName and target_database_JDBC/check_JDBC connection details (Check_* variables are not required for the build stage). A breakdown of why we need these variables can be found below:

• databaseName
  • This is used within the relevant stage, to refer to the database name when required. For example, to be used within the name of a generated report.
• target_database_JDBC
  • A valid JDBC connection string will need to be provided, so that the Flyway CLI can connect to the relevant target stage database.
    • If you're unsure how to setup and test your JDBC connection details, start by following the Flyway Redgate University courses:
      • https://www.red-gate.com/hub/university/courses/flyway
    • Tip - Flyway Desktop can be a great tool for building and testing your JDBC connection strings.
• check_JDBC
  • A valid JDBC connection string will need to be provided, so that the Flyway CLI can connect to the relevant target stage check database. This variable is needed for all stages, EXCEPT the Build stage.
    • The purpose of the Check Database is solely for the ability to create pipeline reports, designed to increase confidence with your migrations (More details can be found in the below link). Therefore, for this variable, it is necessary to provide the JDBC connection details for your designated Check database. Which will be an EMPTY database when it is first created.  
      • https://documentation.red-gate.com/fd/check-167936128.html
        • A single Check database can be used throughout multiple pipeline stages. HOWEVER, it is recommended that a unique check database is created PER STAGE, in order to avoid deployment collisions. I.e multiple Azure DevOps jobs running in parallel, each trying to access the same check database.
    • If you're unsure how to setup and test your JDBC connection details, start by following the Flyway Redgate University courses:
      • https://www.red-gate.com/hub/university/courses/flyway
    • Tip - Flyway Desktop can be a great tool for building and testing your JDBC connection strings.
• check_UserName
  • A valid username for authentication with SQL Server, to connect to the check_JDBC database (Not required for Windows Auth, simply leave the value blank)
• check_password
  • A valid password for authentication with SQL Server, to connect to the check_JDBC database (Not require for Windows Auth, simply leave the value blank)

In order to create variables, simply select the '+ Add' button as many times are required. When entering the value each variable will be associated with, you are also able to select the lock icon to make the value a secret. This is extremely valuable for password variables, which best practice dictates should NEVER be stored as plain text.

Simply copy this process three times, for the three variable groups required by the default pipeline setup, ensuring that the variable group names are as follows (Unless first changed in the pipeline):

• rg_build_credentials_variable_group
• rg_test_credentials_variable_group
• rg_prod_credentials_variable_group

Once complete, it is necessary to create one final variable group, this time called 'flyway_vars'. The goal of this group is to contain the flyway licence key, which can securely be provided to the Flyway CLI where required.

The variables required in this group are as follows:

• AGENT_POOL
  • This should contain the name of the Agent Pool you would like to use throughout the pipeline, said Agent Pool should contain the Self-Hosted agent/s. These will be called upon for use when running the pipeline.
    • The default pipeline behavior is to use a single Agent Pool for whole pipeline. If you require separate agent pools per stage, then this variable will need to be present (With the corresponding Agent Pool) in each of the stage specific variable groups from the steps above. Following this, the YAML pipeline will also need to be updated on lines 71 and 107. Simply change $(AGENT_POOL) to $(stage.AGENT_POOL).
• FLYWAY_LICENSE_KEY
  • Without passing this key/value to the Flyway CLI, only basic Flyway commands can be carried out. I.e Flyway Community actions.
• BASELINE_VERSION
  • This should contain the version number of the baseline script, present in the migrations folder of your Flyway Project.
• FIRST_UNDO_SCRIPT
  • This should contain the version number of the first script that has a corresponding undo script equivalent. This is required so that the pipeline can test the rollback behavior, during the Build stage.
• RELEASE_BRANCH
  • This can be used to set the branch that triggers the pipeline, when new migration scripts are added. For example the Main or Release branch, this will be dependent on your branching naming convention. 

Section 8 - Pipeline Permissions

Upon completion of Section 7, which involved setting up the pipeline and the relevant variables. It is now necessary to carry out your first pipeline run, which will involve setting up pipeline permissions to be able to successfully complete. To start this process, select the 'Run' button from the top-right hand side of the ADO Pipeline editing screen.

You will then be navigate to a screen similar to below, where each stage will sectioned out (According to the pipeline). The pipeline will initially be paused, due to insufficient privileges to both the Variable Groups and Agent Pools. To resolve this, simply select the 'View' button that showcases at the start of each section. Followed by 'Permitting' the permissions for each Variable Group and Agent Pool that is requested.

Should you have been viewing the pipeline from the summary view, the same action is required.

Section 9 - Pipeline Testing

Although this example pipeline has been designed to be simple to stand-up and configure. There may be a need to run the pipeline through a few times, in order to iron out any configuration kinks that may be present. One tip to make this process as simple as possible, is to first test your JDBC connection details using Flyway Desktop. Where it can help build the JDBC connection string syntax, porting this over to the YAML file once complete. If you're unable to access Flyway Desktop, you could also use the Flyway CLI in your preferred Shell to test out the connection details outside of the pipeline.

Section 10 - Conclusion

Flyway is an extremely flexible migrations-first engine, with the CLI able to be built into most modern CI/CD tools. In this article, it showcased how Flyway could be built into an Azure DevOps YAML pipeline. However, this just scratches the surface around how Flyway could be incorporated in almost any pipeline tool to match your business requirements.

Related articles

• Page:
  Welcome To Flyway
• Page:
  GitHub YML Self Hosted Agent (Using GitHub Actions)
• Page:
  Azure DevOps YML Self Hosted Windows Agent
• Page:
  Azure DevOps Docker YML Microsoft Hosted Agent
• Page:
  Jenkins YAML (Jenkinsfile) Windows/Linux