So you want to use an AWS S3 bucket as a location for migrations (or callbacks) ? The existing documentation suggest it's straightforward - Flyway Locations Setting

Sadly, it is only straightforward for a Java development project and much more challenging if you are just using the CLI.

Where to get the dependencies

The AWS SDK isn't released as a zip file or anything so straightforward and requires much more than just a single JAR file be included in Flyway.

If you don't have these installed then you'll get a helpful message from Flyway:

ERROR: Can't read location s3:flyway-testing; AWS SDK not found

Resolving this will require some (limited) Java-fu so if installing a JDK and the Maven build tools is alarming, then this article isn't for you.

Why is it not part of the product ?

At one point these were shipped with Flyway but it added a lot of files and bloat to the product for something that isn't used very often.

It also caused conflicts within user's Java projects so we took the decision to make it an optional part of the shipped product back in Flyway V7

Create a project from an archetype

To get the pieces I needed, I created a AWS sample project from the AWS archetype which is done like this (you may have to adapt this to your region and choice of project name):

mvn -B archetype:generate \
 -DarchetypeGroupId=software.amazon.awssdk \
 -DarchetypeArtifactId=archetype-lambda -Dservice=s3 -Dregion=EU_WEST_1 \
 -DarchetypeVersion=2.38.2 \
 -DgroupId=com.example.myapp \
 -DartifactId=myapp

..and you need some other logging JARs

I also needed to add Apache logging to the project or the resultant set of JAR files won't be enough to make AWS happy - you will need to add this dependency to your project POM file.

        <dependency>
            <groupId>commons-logging</groupId>
            <artifactId>commons-logging</artifactId>
            <version>1.1.1</version>
        </dependency>

Download the dependencies

You can now build the project but by default it creates a shaded JAR so you can't get the jar files you are after out easily.

To do that you need to run this from within your project folder:

mvn dependency:copy-dependencies -DoutputDirectory=target/dependencies -DincludeScope=runtime

This will put all the dependencies you need in the target/dependencies folder so you can use them directly.

Add the dependencies to Flyway

You take the dependencies downloaded in the previous step and copy them into your installation of Flyway drivers/aws folder so they will be picked up when you run Flyway.

The list I got based off AWS SDK archetype version 2.38.2

Configuring your AWS connection

I setup used these environment variables with details from my AWS account

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_SESSION_TOKEN
• AWS_REGION

Testing it out

Assuming you have the connection runes correct and have an S3 bucket set up already called test-bucket then getting Flyway to see it is now straightforward:

./flyway info -url=jdbc:sqlite:dev.db -locations=s3:test-bucket