Using TestContainers for Integration Tests

 Background:

    In CentralCoding service, BDD tests will rely on another database instance. Since there will be a lot of data get imported each time, there's a need to avoid unnecessary cost by using TestContainers.

What is TestContainers

    TestContainers is a java library that provides functionality to handle a docker container. You can start any container by using the GenericContainer with any docker image, one of the specialized containers(e.g PostgresSqlContainer) provided by a module, or by programmatically creating your own image on the fly. TestContainers also ensures that application inside the container is started and ready to use, by providing a WaitStrategy.

Dependency:

    Make sure you have docker installed in your local development.

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter</artifactId>
    <version>5.8.1</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>testcontainers</artifactId>
    <version>1.17.3</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>junit-jupiter</artifactId>
    <version>1.17.3</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>postgressql</artifactId>
    <version>1.17.3</version>
    <scope>test</scope>
</dependency>

TestContainers Usage:

1, Using special JDBC URL(This is specific to Database containers)

2, Auto handling containers using @TestContainers and @Container

3, Manual container starting

By using special JDBC URL, we use jdbc:tc:postgresql:14//localhost/somedb in our case, 

the "tc" will notify the app it's test container and make the container ready, but it 

didn't give us more flexibilities to config the container.

By using @TestContainers and @Container,

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@Testcontainers
@CucumberContextConfiguration
public class CucumberBootstrap {
    @Container
    public static PostgreSQLContainer postgresDB = new PostgreSQLContainer("postgres:14")
            .withDatabaseName("test")
            .withUsername("test")
            .withPassword("test");

    @DynamicPropertySource
    public static void properties(DynamicPropertyRegistry registry) {
        registry.add("multitenancy.master.datasource.url", postgresDB::getJdbcUrl);
        registry.add("multitenancy.tenancy.datasource.url", postgresDB::getJdbcUrl);
        registry.add("spring.datasource.url", postgresDB::getJdbcUrl);
        registry.add("multitenancy.master.datasource.username", postgresDB::getUsername);
        registry.add("spring.datasource.username", postgresDB::getUsername);
        registry.add("multitenancy.master.datasource.password", postgresDB::getPassword);
        registry.add("spring.datasource.password", postgresDB::getPassword);
    }
}

@DynamicPropertySource will dynamically save the database properties in our context
It works fine with @Test, could be an alternative for JPA test, but couldn't figure out
a way to invoke BDD, getting error "Unable to detect database type".

By using manual container starting, utilizes the life cycle of cucumber, @BeforeAll and @AfterAll
@Slf4j
@CucumberContextConfiguration
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@ActiveProfiles("bdd")
public class CucumberBootstrap {
    static PostgreSQLContainer postgresContainer;

    /**
     * Initial database setup
     */
    @BeforeAll
    public static void setup() {
        postgresContainer = new PostgreSQLContainer("postgres:14")
                .withDatabaseName("cc")
                .withUsername("postgres")
                .withPassword("postgres");
        postgresContainer.start();
    }

    /**
     * Datasource dynamic configuration
     *
     */
    @TestConfiguration
    static class PostgresTestConfiguration {
        
        public DataSourceProperties masterDataSourceProperties() {
            DataSourceProperties dataSourceProperties = new DataSourceProperties();
            dataSourceProperties.setUrl(postgresContainer.getJdbcUrl());
            dataSourceProperties.setUsername(postgresContainer.getUsername());
            dataSourceProperties.setPassword(postgresContainer.getPassword());
            dataSourceProperties.setDriverClassName(postgresContainer.getDriverClassName());
            return dataSourceProperties;
        }
        
        @Bean
        @LiquibaseDataSource
        public DataSource masterDataSource() {
            HikariDataSource dataSource = masterDataSourceProperties()
                    .initializeDataSourceBuilder()
                    .type(HikariDataSource.class)
                    .build();
            dataSource.setPoolName("masterDataSource");
            return dataSource;
        }

        public DataSourceProperties tenancyDataSourceProperties() {
            DataSourceProperties dataSourceProperties = new DataSourceProperties();
            dataSourceProperties.setUrl(postgresContainer.getJdbcUrl());
            dataSourceProperties.setUsername("it_user");//role which generated by liquibase 
            dataSourceProperties.setPassword("secret");
            dataSourceProperties.setDriverClassName(postgresContainer.getDriverClassName());
            return dataSourceProperties;
        }

        @Bean
        @Primary
        public DataSource tenancyDataSource() {
            HikariDataSource dataSource = tenancyDataSourceProperties()
                    .initializeDataSourceBuilder()
                    .type(HikariDataSource.class)
                    .build();
            dataSource.setPoolName("tenancyDataSource");
            return new TenancyAwareDataSource(dataSource);
        }
        
        @Bean
        @Primary
        public TransactionManager primaryTransactionManager(@Qualifier("transactionManager") 
                                                                            TransactionManager transactionManager){
            return transactionManager;
        }
    }

    /**
     * Shutdown
     */
    @AfterAll
    public static void tearDown() {
        System.out.println("closing DB connection");
        postgresContainer.stop();
    }
}

Since CC uses liquidate to take care of db scripts and also use the RLS(row level security)
feature in Postgres, multiple datasources add the complexity to this POC. In order to control
the datasource, @ActiveProfiles("bdd") and @TestConfiguration are being used. So BDD will
use "bdd" profile while DataSourceConfiguration will be used in not "bdd" profiles.

@Configuration
@Profile("bdd")
public class DataSourceConfiguration {

    @Bean
    @ConfigurationProperties("multitenancy.master.datasource")
    public DataSourceProperties masterDataSourceProperties() {
        return new DataSourceProperties();
    }

    @Bean
    @LiquibaseDataSource
    @ConfigurationProperties("multitenancy.master.datasource.hikari")
    public DataSource masterDataSource() {
        HikariDataSource dataSource = masterDataSourceProperties()
                .initializeDataSourceBuilder()
                .type(HikariDataSource.class)
                .build();
        dataSource.setPoolName("masterDataSource");
        return dataSource;
    }

    @Bean
    @Primary
    @ConfigurationProperties("multitenancy.tenancy.datasource")
    public DataSourceProperties tenancyDataSourceProperties() {
        return new DataSourceProperties();
    }

    @Bean
    @Primary
    @ConfigurationProperties("multitenancy.tenancy.datasource.hikari")
    public DataSource tenancyDataSource() {
        HikariDataSource dataSource = tenancyDataSourceProperties()
                .initializeDataSourceBuilder()
                .type(HikariDataSource.class)
                .build();
        dataSource.setPoolName("tenancyDataSource");
        return new TenancyAwareDataSource(dataSource);
    }

    @Bean
    @Primary
    public TransactionManager primaryTransactionManager(@Qualifier("transactionManager")
                                                                TransactionManager transactionManager){
        return transactionManager;
    }
}

In general, testing with TestContainers will take longer time.

References:
https://medium.com/javarevisited/cucumber-testcontainer-a-bdd-perfect-match-956cf62cdf47
https://fullstackcode.dev/2021/05/16/springboot-integration-testing-with-testcontainers/