Redgate Flyway

Tutorial - Update development environment automatically on branch switch using Redgate Clone (git hooks)

This approach requires a lot more manual steps than using the Redgate Clone provisioner and will generally be less preferable. The main difference is that this will provision a new database as part of the git branch switch process, as opposed to when you first connect to the database on the new branch,


  1. Install the Redgate Clone command line on your desktop and/or agents to use for Dev and Test use cases.
  2.  Create a helper-files folder in your local repository and save the following 3 files there.

    post-checkout Toggle source code

    1. #!/bin/sh
    2. #exec powershell.exe -NoProfile -ExecutionPolicy Bypass -File "<TODO_RELATIVE_PATH>\clone.sh"
    3.  
    4.  
    5. exec "<TODO_RELATIVE_PATH>\clone.sh"
    6.  
    7. exit

    clone.sh Toggle source code

    1. #!/bin/bash
    2. DATA_IMAGE="<TODO_IMAGE_NAME>"
    3. FW_KEY="<TODO_LICENSE_KEY>"
    4. BRANCH_NAME="$(git rev-parse --symbolic-full-name --abbrev-ref HEAD)"
    5. CONTAINER_NAME="<TODO_PROJECT_NAME>.$BRANCH_NAME.$USERNAME"
    6. STANDBY_NAME="<TODO_PROJECT_NAME>.standby"
    7. DEV_PORT="10000"
    8.  
    9. # Timing the total operation
    10. totalStartTime=$(date +%s)
    11.  
    12. # In case the proxy isn't running/working, we kill it here and reinstate it later
    13. echo "Kill any rgclone proxy that may be running"
    14. tskill rgclone
    15.  
    16. if ! rgclone.exe get data-container "$CONTAINER_NAME" &> /dev/null ;
    17. then
    18. echo "No data container for the current branch - will try to provision one"
    19. if ! rgclone.exe get dc $STANDBY_NAME &> /dev/null ;
    20. then
    21. echo "No standby found - please wait a couple of minutes while we create a new clone (and a standby for future use)"
    22.  
    23. startTime=$(date +%s)
    24.  
    25. sh $PWD/clone-standby.sh
    26.  
    27. # Now we have an up-to-date migrated standby we can rename immediately for our branch
    28. rgclone.exe update dc $STANDBY_NAME -n "$CONTAINER_NAME" --lifetime 30m
    29. echo "Standby data container has been renamed to $CONTAINER_NAME"
    30. endTime=$(date +%s)
    31. echo ""
    32. echo "######## DATA CONTAINER CREATED ###############"
    33. echo "Time taken: $(expr $endTime - $startTime) seconds"
    34.  
    35. else
    36. echo "We have found a standby that we can update and rename for our new branch!"
    37. # Update the standby in case pending migrations have been created since it was created
    38. echo "Running Flyway migrate on standby"
    39. CLONE_PASSWORD="$(rgclone.exe get data-container $STANDBY_NAME -o json | jq .password -j)"
    40. PORT="$(rgclone.exe get data-container $STANDBY_NAME -o json | jq .port -j)"
    41. HOST="$(rgclone.exe get data-container $STANDBY_NAME -o json | jq .host -j)"
    42. URL="jdbc:sqlserver://$HOST:$PORT;databaseName=StackOverflow;encrypt=true;trustServerCertificate=true;"
    43.  
    44. echo "Running Flyway info migrate: $URL port $PORT host $HOST pass $CLONE_PASSWORD"
    45. flyway info migrate -url=$URL -user=sa -password=$CLONE_PASSWORD info -licenseKey=$FW_KEY
    46. rgclone.exe update dc $STANDBY_NAME -n "$CONTAINER_NAME" --lifetime 30m
    47. echo "Standby data container has been renamed to $CONTAINER_NAME"
    48. fi
    49.  
    50. # need to update the state from the schema model
    51. # We have to restart the proxy to use flyway-dev as this pulls the connection details from flyway-dev.json
    52. echo "Now we start the proxy on port $DEV_PORT"
    53. rgclone.exe proxy dc $CONTAINER_NAME --port $DEV_PORT &
    54. echo "Updating the schema state (this will overwrite any changes you haven't committed)"
    55. flyway-dev diff --project $PWD --from SchemaModel --to Dev --i-agree-to-the-eula --artifact changes.zip
    56. echo "Running Take and piping to Apply (through less as I can't get it working without this)"
    57. flyway-dev take --artifact changes.zip --project $PWD --i-agree-to-the-eula | less | flyway-dev apply --artifact changes.zip --project $PWD --i-agree-to-the-eula
    58. rm changes.zip
    59. echo "#####################################################"
    60. echo ""
    61. echo "###### DEV DB up to date and ready to use! ##########"
    62. echo ""
    63. echo "## (although a new standby will be kicked off now) ##"
    64. echo "#####################################################"
    65. # Now we should create a new standby for the next branch/person
    66. sh $PWD/clone-standby.sh &> /dev/null
    67.  
    68. else
    69. echo ""
    70. echo "A data container already exists for the branch $CONTAINER_NAME"
    71. echo ""
    72. echo "We reset the lifetime of the container to 30m"
    73. rgclone update dc $CONTAINER_NAME --lifetime 30m
    74.  
    75. echo "Now we start the proxy on port $DEV_PORT"
    76. rgclone.exe proxy dc $CONTAINER_NAME --port $DEV_PORT &
    77. fi
    78.  
    79. totalEndTime=$(date +%s)
    80.  
    81. echo "###########################################"
    82. echo ""
    83. echo "######## ALL DONE !!! #####################"
    84. echo "Time taken: $(expr $totalEndTime - $totalStartTime) seconds"
    85. echo "###########################################"

    clone-standby.sh Toggle source code

    1. #!/bin/bash
    2. IMAGE="<TODO_IMAGE_NAME>"
    3. KEY="<TODO_LICENSE_kEY>"
    4. BRANCH_NAME="$(git rev-parse --symbolic-full-name --abbrev-ref HEAD)"
    5. CONTAINER_NAME="<TODO_PROJECT_NAME>.$BRANCH_NAME.$USERNAME"
    6. DEV_PORT="10000"
    7. LIFETIME="240m"
    8. CONTAINER="<TODO_PROJECT_NAME>.standby"
    9.  
    10. echo ""
    11. echo "Creating a brand new data container based on $IMAGE"
    12. echo ""
    13. echo "######## Data Container CREATED ###############"
    14.  
    15. rgclone.exe create dc --name "$CONTAINER" --image "$IMAGE" --lifetime $LIFETIME
    16.  
    17. CLONE_PASSWORD="$(rgclone.exe get data-container $CONTAINER -o json | jq .password -j)"
    18. PORT="$(rgclone.exe get data-container $CONTAINER -o json | jq .port -j)"
    19. HOST="$(rgclone.exe get data-container $CONTAINER -o json | jq .host -j)"
    20. #Ideally all in the following section are pre-configured in the data image
    21. # TODO - try to combine these to make this part faster
    22. URL="jdbc:sqlserver://$HOST:$PORT;databaseName=StackOverflow;encrypt=true;trustServerCertificate=true;"
    23. echo "URL is: $URL"
    24. flyway -initSql="EXEC sp_addrolemember 'db_owner', 'Dev1'" -url=$URL -user=sa -password=$CLONE_PASSWORD info -licenseKey=$KEY -connectRetries=7
    25.  
    26. echo "Updating Server adding Dev1 to sysadmin role - required for tSQLt"
    27. echo "Enabling CLR - required for tSQLt"
    28. echo "Enabling Advanced options so strict security can be disabled - required for tSQLt"
    29. echo "Disabling strict security - required for tSQLt"
    30. URL="jdbc:sqlserver://$HOST:$PORT;databaseName=Master;encrypt=true;trustServerCertificate=true;"
    31. flyway -initSql="ALTER SERVER ROLE [sysadmin] ADD MEMBER [Dev1];EXEC sp_configure 'clr enabled', 1; EXEC sp_configure 'show advanced options', 1; RECONFIGURE; EXEC sp_configure 'clr strict security', 0; RECONFIGURE;" -url=$URL -user=sa -password=$CLONE_PASSWORD info -licenseKey=$KEY
    32.  
    33. echo "Baselining dev DB"
    34. echo "Migrating dev DB to the most recent version"
    35. URL="jdbc:sqlserver://$HOST:$PORT;databaseName=StackOverflow;encrypt=true;trustServerCertificate=true;"
    36. flyway baseline migrate -url=$URL -user=sa -password=$CLONE_PASSWORD info -licenseKey=$KEY
    37.  
    38. # need to update the state from the schema model
    39. # but can't as this only works when the proxy is on, as this uses the flyway-dev.json dev connection string
    40. #flyway-dev diff --project $PWD --from SchemaModel --to Dev --i-agree-to-the-eula --artifact changes.zip
    41. #echo "Running Take and piping to Apply (through less)"
    42. #flyway-dev take --artifact changes.zip --project $PWD --i-agree-to-the-eula | less | flyway-dev apply --artifact changes.zip --project $PWD --i-agree-to-the-eula
    43. #rm changes.zip
    44.  
    45. #################################################
    46.  
    47. echo "Updating Shadow DB"
    48. URL="jdbc:sqlserver://$HOST:$PORT;databaseName=StackOverflow_Shadow;encrypt=true;trustServerCertificate=true;"
    49. flyway -initSql="EXEC sp_addrolemember 'db_owner', 'Dev1'" -url=$URL -user=sa -password=$CLONE_PASSWORD info -licenseKey=$KEY
    50.  
    51.  
  3. Copy the post-checkout file to the .git\hooks folder in your local repo .  Each developer will need to copy this to their own local repo.
  4. Make sure the path to rgclone.exe and flyway are on your Systems Environment Variables Path so the post-checkout script can find them.  You may need to restart your computer after adding these.
  5. You'll also need jq.
    1. Run cmd as admin and run chocolatey install jq.
  6. Switch to the Version Control  tab and create a new branch from dev.  This will use the post-checkout hook to call the clone-branch.sh script to spin up a new dedicated development environment for you on that branch.  The shell script also updates your localproxy to point to this new environment.  (Sometimes you may need to refresh in FWD if the db connection time out.)
  7. If you want to work on main, just switch your branch back to main and a dedicated environment for main will be created.

Didn't find what you were looking for?