git reset/revert to previous commit

Sometimes we "accidentally" merge something that's not ready for release/qa  and need to rollback/reset/revert the commits to a cleaner state.

Here's how you can safely reset your git to previous commit:

1) find the older commit id to where you want to reset.

2) do git status to confirm there are no uncommitted changes on the current branch  

3) reset to a commit,

git reset --hard COMMIT_ID
git reset HEAD@{1}
git add .
git commit -m "reverting to commit COMMIT_ID"

4) cherry pick or patch any other commit that you like to add


How to make integration tests faster without @DirtiesContext

If your only excuse to use @DirtiesContext is to re-initialize database between tests, then this blog post is for you.

Spring's @DirtiesContext is very useful to make your integration tests faster by indicating the underlying Spring ApplicationContext is modified and forcing it to reload the context (not the whole application) between tests.

This is particularly useful when we want to reset the state of database between tests. This allows us to group many tests together in a same class so that we can easily share some test data, assertions etc. So your test code would look this this:

@SpringBootTest
@DirtiesContext(classMode = AFTER_EACH_TEST_METHOD)
class SomeTest {

@Autowired SomeService serv;
@Autowired DataCreator dc;

@Test
void someTest1() {
//prepare data
dc.initData();
//run test
serv.modifySomeData();
//verify
}

@Test
void someTest2() {
//prepare data
dc.initData(2);
//run test
serv.modifyAnotherData();
serv.doSomethingElse();
//verify
}
}

Problem with @DirtiesContext

This has one major flaw though. By resetting the ApplicationContext, it would also drop all the databases and need to re-create them after every test. So, if the only reason to use @DirtiesContext is to reset the database, then there's a better way of doing this.

Is deleting records from all table not a good solution?

Correct. Deleting records is still a slow operation. When you have a good amount to test data, the @DirtiesContext can sometimes becomes faster than deleting the records one by one. Also, there's referential integrity between tables that enforces you to follow a series of deletes to first delete records from child tables and go up. 

Table Truncate to the rescue

Truncate (truncate table X) is faster way to delete records from database than deleting records. If we already know the list of table and join tables then we can loop through them and execute the TRUNCATE TABLE command. One advantage of this approach is that we can skip truncating some lookup tables that will always have constant records.

How to Truncate H2 Tables:

Note that before we execute the TRUNCATE TABLE we should disable the referential integrity.
em.createNativeQuery("SET REFERENTIAL_INTEGRITY FALSE").executeUpdate();
for (String t : tableNames) {
em.createNativeQuery("TRUNCATE TABLE " + t ).executeUpdate();
}
em.createNativeQuery("SET REFERENTIAL_INTEGRITY TRUE").executeUpdate();

Integrating everything together (Sample App @ GitHub):

TestDataManager to grab list of tables, truncate and populate the test data:

import gt.app.DataCreator;
import gt.app.config.MetadataExtractorIntegrator;
import lombok.RequiredArgsConstructor;
import org.hibernate.boot.Metadata;
import org.hibernate.mapping.Collection;
import org.hibernate.mapping.PersistentClass;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import javax.persistence.EntityManager;
import java.util.ArrayList;
import java.util.List;

@Service
@RequiredArgsConstructor
public class TestDataManager implements InitializingBean {
final EntityManager em;
final DataCreator dataCreator;

private final List<String> tableNames = new ArrayList<>(); //shared

@Override
public void afterPropertiesSet() {
Metadata metadata = MetadataExtractorIntegrator.INSTANCE.getMetadata();

for (Collection persistentClass : metadata.getCollectionBindings()) {
tableNames.add(persistentClass.getCollectionTable().getExportIdentifier());
}

for (PersistentClass persistentClass : metadata.getEntityBindings()) {
tableNames.add(persistentClass.getTable().getExportIdentifier());
}
}

@Transactional
public void truncateTablesAndRecreate() {
truncateTables();
dataCreator.initData();
}

void truncateTables() {
em.createNativeQuery("SET REFERENTIAL_INTEGRITY FALSE").executeUpdate();
for (String tableName : tableNames) {
em.createNativeQuery("TRUNCATE TABLE " + tableName).executeUpdate();
}
em.createNativeQuery("SET REFERENTIAL_INTEGRITY TRUE").executeUpdate();
}
}

Call the truncateTablesAndRecreate() @BeforeEach test

@SpringBootTest
class WebAppIT {

@Autowired
TestDataManager tdm;

@BeforeEach
void resetDB(){
tdm.truncateTablesAndRecreate();
}

 

This is awesome, how about other databases?

To get a List of tables: You can rely on Hibernate's Metadata to grab list of table/join tables

Truncate command: Each database has different command to set the referential integrity and truncate. This above example is for H2.

You can do the following for MySQL


//for MySQL:
em.createNativeQuery("SET @@foreign_key_checks = 0").executeUpdate();
for (String tableName : tableNames) {
em.createNativeQuery("TRUNCATE TABLE " + tableName).executeUpdate();
}
em.createNativeQuery("SET @@foreign_key_checks = 1").executeUpdate();

For other populate databases, you can take reference from Hibernate's Test code itself. They are available at: Hibernate GitHub. The *Cleaner classes have code snippet that describes how to perform truncate on each database.

How fast the tests ran after this update?

Very fast! In a big application (Spring Boot, H2) with 62 tables and 315 tests that were using @DirtiesContext, we reduced our test execution time from 18 minutes to 4minutes.

Full working example is available at this sample app.

JPA/Hibernate get find all Table and Column metadata

How to use Hibernate Metadata to find All columns and tables

Getting the Hibernate's Metadata object into the Spring application is tricky. Luckily, Hibernate Provides an Integrator API (org.hibernate.integrator.spi.Integrator) that we can use to customize/interact with Hibernate. Its the same API that Caching, Bean Validation etc library uses to integrate with Hibernate. 

Also, Spring Boot provides HibernatePropertiesCustomizer to link the 'hibernate.integrator_provider' property to Integrator implementation.

Here's how we can configure the Hibernate Integrator to read metadata.

Step 1) create a extractor implementation

This class is a singleton class and does absolutely nothing other than exposing the Metadata and Database. Since this is singleton this class and the database, metadata objects can be statically accessed using MetadataExtractorIntegrator.INSTANCE

import lombok.Data;
import org.hibernate.boot.Metadata;
import org.hibernate.boot.model.relational.Database;
import org.hibernate.engine.spi.SessionFactoryImplementor;
import org.hibernate.integrator.spi.Integrator;
import org.hibernate.service.spi.SessionFactoryServiceRegistry;

@Data
public class MetadataExtractorIntegrator implements Integrator {

public static final MetadataExtractorIntegrator INSTANCE =
new MetadataExtractorIntegrator();
private Database database;
private Metadata metadata;

@Override
public void integrate(Metadata metadata, SessionFactoryImplementor sf,
SessionFactoryServiceRegistry sr) {
this.database = metadata.getDatabase();
this.metadata = metadata;
}

@Override
public void disintegrate(SessionFactoryImplementor sf,
SessionFactoryServiceRegistry sr) {
}
}

Step 2) Register the Spring Hibernate Customizer


import org.hibernate.jpa.boot.spi.IntegratorProvider;
import org.springframework.boot.autoconfigure.orm.jpa.HibernatePropertiesCustomizer;
import org.springframework.context.annotation.Configuration;

import java.util.List;
import java.util.Map;

@Configuration
public class HibernateConfig implements HibernatePropertiesCustomizer {
@Override
public void customize(Map<String, Object> hibernateProps) {
hibernateProps.put("hibernate.integrator_provider",
(IntegratorProvider) () -> List.of(MetadataExtractorIntegrator.INSTANCE));
}
}

Step 3) Use MetadataExtractorIntegrator.metadata to extract the metadata

This is a simple Spring Component that uses Metadata.getCollectionBindings and Metadata.getEntityBindings to extract the tables, columns, PK and type

import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.hibernate.boot.Metadata;
import org.hibernate.mapping.*;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.stereotype.Component;

import javax.persistence.EntityManager;
import java.util.Iterator;

@Component
@RequiredArgsConstructor
@Slf4j
public class DBMetadataReader implements InitializingBean {
final EntityManager em;

@Override
public void afterPropertiesSet() {

Metadata metadata = MetadataExtractorIntegrator.INSTANCE.getMetadata();

//Collection tables
for (Collection c : metadata.getCollectionBindings()) {
log.info("Collection table: {}", c.getCollectionTable().getQualifiedTableName());
for (Iterator<Column> it = c.getCollectionTable().getColumnIterator();
it.hasNext(); ) {
Column property = it.next();
log.info(" {} {} ", property.getName(), property.getSqlType());
}
}

//all entities
for (PersistentClass pc : metadata.getEntityBindings()) {
Table table = pc.getTable();

log.info("Entity: {} - {}", pc.getClassName(), table.getName());

KeyValue identifier = pc.getIdentifier();

//PK
for (Iterator<Selectable> it = identifier.getColumnIterator();
it.hasNext(); ) {
Column column = (Column) it.next();
log.info(" PK: {} {}", column.getName(), column.getSqlType());
}

//property/columns
for (Iterator it = pc.getPropertyIterator();
it.hasNext(); ) {
Property property = (Property) it.next();

for (Iterator columnIterator = property.getColumnIterator();
columnIterator.hasNext(); ) {
Column column = (Column) columnIterator.next();
log.info(" {} {}", column.getName(), column.getSqlType());
}
}
}
}
}

Example Project:

Checkout my sample github project and the source for working example