Spring Boot MongoDB Migrations using Mongock

January 16, 2021

Spring Boot MongoDB Migrations using Mongock

In this article, we are going to learn how to perform database migrations for your Spring Boot applications backed by MongoDB using a library called Mongock.

Why we need Database Migrations ?

If you are working on any non-trivial project, chances are over the span of time, you will add new features and functionalities, and your domain model will also evolve along with it.

For example, we may have to add/update a field or document inside a collection in your database across all environments, doing this manually is a boring and error prone process, and most of all its hard to keep track of your changes.

Database migration tools like Mongock helps us to automate the above mentioned process.

If you are coming from a Relational Database world, Mongock is very similar to Flyway or Liquibase.

Download Source Code

You can download the example source code for this tutorial from this link

Exploring the sample project

The project we are going to have a look at is simple REST API used to track Expenses. It’s written using Spring Boot with MongoDB as the database.

If you want to learn how to build the REST API using Spring Boot and MongoDB, have a look at this written tutorial.

Here is the contents of the pom.xml file

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.4.1</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.programming.techie</groupId>
    <artifactId>spring-boot-mongodb-tutorial</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>spring-boot-mongodb-tutorial</name>
    <description>Demo project for Spring Boot</description>

    <properties>
        <java.version>15</java.version>
        <testcontainers.version>1.15.1</testcontainers.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-mongodb</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>junit-jupiter</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>mongodb</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <excludes>
                        <exclude>
                            <groupId>org.projectlombok</groupId>
                            <artifactId>lombok</artifactId>
                        </exclude>
                    </excludes>
                </configuration>
            </plugin>
        </plugins>
    </build>

</project>

Let’s have a look at the domain object Expense.java

package com.programming.techie.mongo.model;

import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import org.springframework.data.annotation.Id;
import org.springframework.data.mongodb.core.index.Indexed;
import org.springframework.data.mongodb.core.mapping.Document;
import org.springframework.data.mongodb.core.mapping.Field;

import java.math.BigDecimal;

@Getter
@Setter
@AllArgsConstructor
@NoArgsConstructor
@Document("expense")
public class Expense {
    @Id
    private String id;
    @Field("name")
    @Indexed(unique = true)
    private String expenseName;
    @Field("category")
    private ExpenseCategory expenseCategory;
    @Field("amount")
    private BigDecimal expenseAmount;
}

ExpenseCategory.java

package com.programming.techie.mongo.model;

public enum ExpenseCategory {
    ENTERTAINMENT, GROCERIES, RESTAURANT, UTILITIES, MISC
}

Here is the REST Controller, ExpenseRestController.java

package com.programming.techie.mongo.controller;

import com.programming.techie.mongo.model.Expense;
import com.programming.techie.mongo.service.ExpenseService;
import lombok.RequiredArgsConstructor;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;

import java.util.List;

@RestController
@RequestMapping("/expense")
@RequiredArgsConstructor
public class ExpenseController {

    private final ExpenseService expenseService;

    @PostMapping
    public ResponseEntity addExpense(@RequestBody Expense expense) {
        expenseService.addExpense(expense);
        return ResponseEntity.status(HttpStatus.CREATED).build();
    }

    @PutMapping
    public ResponseEntity updateExpense(@RequestBody Expense expense) {
        expenseService.updateExpense(expense);
        return ResponseEntity.status(HttpStatus.OK).build();
    }

    @GetMapping
    public ResponseEntity<List<Expense>> getAllExpenses() {
        return ResponseEntity.ok(expenseService.getAllExpenses());
    }

    @GetMapping("/{name}")
    public ResponseEntity getExpenseByName(@PathVariable String name) {
        return ResponseEntity.ok(expenseService.getExpense(name));
    }

    @DeleteMapping("/{id}")
    public ResponseEntity deleteExpense(@PathVariable String id) {
        expenseService.deleteExpense(id);
        return ResponseEntity.noContent().build();
    }

}

ExpenseService.java

package com.programming.techie.mongo.service;

import com.programming.techie.mongo.model.Expense;
import com.programming.techie.mongo.repository.ExpenseRepository;
import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import java.util.List;

@Service
@RequiredArgsConstructor
@Transactional
public class ExpenseService {

    private final ExpenseRepository expenseRepository;

    public void addExpense(Expense expense) {
        expenseRepository.insert(expense);
    }

    public void updateExpense(Expense expense) {
        Expense savedExpense = expenseRepository.findById(expense.getId()).orElseThrow(() -> new RuntimeException(String.format("Cannot Find Expense by ID %s", expense.getId())));
        savedExpense.setExpenseName(expense.getExpenseName());
        savedExpense.setExpenseCategory(expense.getExpenseCategory());
        savedExpense.setExpenseAmount(expense.getExpenseAmount());

        expenseRepository.save(expense);
    }

    public Expense getExpense(String name) {
        return expenseRepository.findByName(name)
                .orElseThrow(() -> new RuntimeException(String.format("Cannot Find Expense by Name - %s", name)));
    }

    public List<Expense> getAllExpenses() {
        return expenseRepository.findAll();
    }

    public void deleteExpense(String id) {
        expenseRepository.deleteById(id);
    }
}

ExpenseRepository.java

package com.programming.techie.mongo.repository;

import com.programming.techie.mongo.model.Expense;
import org.springframework.data.mongodb.repository.MongoRepository;
import org.springframework.data.mongodb.repository.Query;

import java.util.Optional;

public interface ExpenseRepository extends MongoRepository<Expense, String> {
    @Query("{'name': ?0}")
    Optional<Expense> findByName(String name);
}

Install Mongock in our Project

To install Mongock library in our project, we have to download the following dependencies into our pom.xml file.

mongock-bom

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>com.github.cloudyrock.mongock</groupId>
            <artifactId>mongock-bom</artifactId>
            <version>4.1.19</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

mongock-spring-v5

<dependency>
    <groupId>com.github.cloudyrock.mongock</groupId>
    <artifactId>mongock-spring-v5</artifactId>
</dependency>

mongodb-springdata-v3-driver

<dependency>
    <groupId>com.github.cloudyrock.mongock</groupId>
    <artifactId>mongodb-springdata-v3-driver</artifactId>
</dependency>

Enable Mongock

To Enable Mongock in our application, we have to follow the below steps:

Adding @EnableMongock annotation to our Application class (SpringBootMongodbTutorialApplication.java)
Add the package which contains the ChangeLog classes to the mongock.change-logs-scan-package property inside the application.properties file

SpringBootMongodbTutorialApplication.java

package com.programming.techie.mongo;

import com.github.cloudyrock.spring.v5.EnableMongock;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
@EnableMongock
public class SpringBootMongodbTutorialApplication {

    public static void main(String[] args) {
        SpringApplication.run(SpringBootMongodbTutorialApplication.class, args);
    }
}

application.properties

####### Mongo Properties ###########
spring.data.mongodb.uri=mongodb://localhost:27017/expense-tracker
####### Mongock Properties ###########
mongock.change-logs-scan-package=com.programming.techie.mongo.config

In the above file, the package com.programming.techie.mongo.config contains the ChangeLog classes to perform the Database Migrations.

ChangeLogs and ChangeSets

Mongock uses ChangeLogs which are the Java Classes responsible for Migrations and ChangeSets, which are methods which are responsible to apply the migrations to the database schema.

To be able to run migrations,

We have to annotate the class responsible for migration with @ChangeLog
And annotate the method responsible to apply migrations with @Changeset annotation.

At the time of startup, Mongock scans the package provided for the mongock.change-logs-scan-package for the ChangeLog and Changesets, and will start executing it.

Let’s create the java class responsible for the migration – DatabaseChangeLog.java

package com.programming.techie.mongo.config;

import com.github.cloudyrock.mongock.ChangeLog;
import com.github.cloudyrock.mongock.ChangeSet;
import com.programming.techie.mongo.model.Expense;
import com.programming.techie.mongo.model.ExpenseCategory;
import com.programming.techie.mongo.repository.ExpenseRepository;

import java.math.BigDecimal;
import java.util.ArrayList;
import java.util.List;

import static com.programming.techie.mongo.model.ExpenseCategory.*;

@ChangeLog
public class DatabaseChangeLog {

    @ChangeSet(order = "001", id = "seedDatabase", author = "Sai")
    public void seedDatabase(ExpenseRepository expenseRepository) {
        List<Expense> expenseList = new ArrayList<>();
        expenseList.add(createNewExpense("Movie Tickets", ENTERTAINMENT, BigDecimal.valueOf(40)));
        expenseList.add(createNewExpense("Dinner", RESTAURANT, BigDecimal.valueOf(60)));
        expenseList.add(createNewExpense("Netflix", ENTERTAINMENT, BigDecimal.valueOf(10)));
        expenseList.add(createNewExpense("Gym", MISC, BigDecimal.valueOf(20)));
        expenseList.add(createNewExpense("Internet", UTILITIES, BigDecimal.valueOf(30)));

        expenseRepository.insert(expenseList);
    }

    private Expense createNewExpense(String expenseName, ExpenseCategory expenseCategory, BigDecimal amount) {
        Expense expense = new Expense();
        expense.setExpenseName(expenseName);
        expense.setExpenseAmount(amount);
        expense.setExpenseCategory(expenseCategory);
        return expense;
    }
}

Here you can see that the class is annotated with @ChangeLog annotation, which indicates that it is a class Mongock should run for applying migrations.
We created a method called seedDatabase() and annotated it with @ChangeSet annotation.
The @ChangeSet takes in some properties called order which decides the order in which the migration should be applied to the database schema.
id to be able to uniquely identify a ChangeSet
author to be able to set the name of the author who created this Changeset.

Inside the ChangeSet we are seeding MongoDB with some test data, and you can see that I am using the ExpenseRepository interface to save the data, which is injected into the method by Spring.

In our Example we are using the Spring Data Repository to save the data to database but Mongock provides different ways of applying the ChangeSets, you can refer the documentation for more details.

Now let’s start the application and see if Mongock is able to seed the MongoDB with our test data or not.

By having a look at the logs, we can see that Mongock applied the ChangeSet successfully to MongoDB. Let’s verify by querying the database.

use expense-tracker;
db.getCollection("expense").find({});

This should return the following output:

{ 
    "_id" : ObjectId("5ff6303090b13917dfb170b7"), 
    "name" : "Movie Tickets", 
    "category" : "ENTERTAINMENT", 
    "amount" : "40", 
    "_class" : "com.programming.techie.mongo.model.Expense"
}
{ 
    "_id" : ObjectId("5ff6303090b13917dfb170b8"), 
    "name" : "Dinner", 
    "category" : "RESTAURANT", 
    "amount" : "60", 
    "_class" : "com.programming.techie.mongo.model.Expense"
}
{ 
    "_id" : ObjectId("5ff6303090b13917dfb170b9"), 
    "name" : "Netflix", 
    "category" : "ENTERTAINMENT", 
    "amount" : "10", 
    "_class" : "com.programming.techie.mongo.model.Expense"
}
{ 
    "_id" : ObjectId("5ff6303090b13917dfb170ba"), 
    "name" : "Gym", 
    "category" : "MISC", 
    "amount" : "20", 
    "_class" : "com.programming.techie.mongo.model.Expense"
}
{ 
    "_id" : ObjectId("5ff6303090b13917dfb170bb"), 
    "name" : "Internet", 
    "category" : "UTILITIES", 
    "amount" : "30", 
    "_class" : "com.programming.techie.mongo.model.Expense"
}

How Mongock manages Database Migrations?

Mongock also keeps track of the ChangeLog, by storing them inside the mongockChangeLog collection, once a ChangeSet is applied, it marks it as EXECUTED to prevent re-running this migration once again.

use expense-tracker;
db.getCollection("mongockChangeLog").find({});

{ 
    "_id" : ObjectId("5ff6303090b13917dfb170bc"), 
    "executionId" : "2021-01-06T22:48:32.481939600-722", 
    "changeId" : "seedDatabase", 
    "author" : "Sai", 
    "timestamp" : ISODate("2021-01-06T21:48:32.520+0000"), 
    "state" : "EXECUTED", 
    "changeLogClass" : "com.programming.techie.mongo.config.DatabaseChangeLog", 
    "changeSetMethod" : "seedDatabase", 
    "executionMillis" : NumberLong(36), 
    "_class" : "io.changock.driver.api.entry.ChangeEntry"
}

It stores the changeId, changeSetMethod and the changeLogClass details among other details of the ChangeLog.

Conclusion

This is the end of this tutorial, I hope you learned how to perform database migrations for MongoDB using Mongock.

I will see you in the next tutorial, until then Happy Coding 🙂

Share0

Tweet0

About the author

Sai Upadhyayula

Farzain Pathan says:

January 23, 2021 at 10:21 am

Hi Sai,

First, thank you for such a detailed and wonderful blog.
But, when I was trying I stuck with the below error:
I using embedded MongoDB

[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 9.754 s <<< FAILURE! – in com.product.target.bootstrsp.TargetApplicationTests
[ERROR] contextLoads Time elapsed: 0 s <<< ERROR!
java.lang.IllegalStateException: Failed to load ApplicationContext
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'mongockBuilder' defined in class path resource [com/github/cloudyrock/spring/v5/MongockSpringDataV3CoreContext.class]: Bean instantiation via factory method failed; n
ested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.github.cloudyrock.spring.v5.MongockSpring5$Builder]: Factory method 'mongockBuilder' threw exception; nested exception is java.lang.NullPointerException: Cannot invo
ke "java.util.List.iterator()" because "changeLogsScanPackage" is null
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.github.cloudyrock.spring.v5.MongockSpring5$Builder]: Factory method 'mongockBuilder' threw exception; nested exception is java.lang.NullPointerException: Cannot invoke "java
.util.List.iterator()" because "changeLogsScanPackage" is null
Caused by: java.lang.NullPointerException: Cannot invoke "java.util.List.iterator()" because "changeLogsScanPackage" is null
1. Sai Upadhyayula says:
  
  January 25, 2021 at 8:28 pm
  
  Hi,
  
  It says changeLogsScanPackage is null, I guess you did not add the following property to the application.properties file:
  
  mongock.change-logs-scan-package=<Your Package Name which contains the Change Log Class>
  1. Farzain Pathan says:
    
    January 29, 2021 at 9:50 pm
    
    Hi Sai,
    
    I already added that property in the application.properties file.
  2. Farzain Pathan says:
    
    January 29, 2021 at 9:52 pm
    
    Hi Sai,
    
    Can you please share your GIT repo link, so that I can at least cross-verify things and get going?
    I would really appreciate that because this the only useful information I found till now on this topic.
Farzain Pathan says:

January 23, 2021 at 10:38 am

Hi Sai,

First, thank you for such a detailed and wonderful blog.
But, when I was trying I am stuck with the below error:
I am using embedded MongoDB

[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 9.754 s <<< FAILURE! – in com.product.target.bootstrsp.TargetApplicationTests
[ERROR] contextLoads Time elapsed: 0 s <<< ERROR!
java.lang.IllegalStateException: Failed to load ApplicationContext
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'mongockBuilder' defined in class path resource [com/github/cloudyrock/spring/v5/MongockSpringDataV3CoreContext.class]: Bean instantiation via factory method failed; n
ested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.github.cloudyrock.spring.v5.MongockSpring5$Builder]: Factory method 'mongockBuilder' threw exception; nested exception is java.lang.NullPointerException: Cannot invo
ke "java.util.List.iterator()" because "changeLogsScanPackage" is null
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.github.cloudyrock.spring.v5.MongockSpring5$Builder]: Factory method 'mongockBuilder' threw exception; nested exception is java.lang.NullPointerException: Cannot invoke "java
.util.List.iterator()" because "changeLogsScanPackage" is null
Caused by: java.lang.NullPointerException: Cannot invoke "java.util.List.iterator()" because "changeLogsScanPackage" is null

Comments are closed.