Spring data Pagination - set max page size and other customizations

Background:

HandlerMethodArgumentResolver is a strategy interface to resolve method parameters in context of given context. So, if you want to automatically resolve the parameter MyObject in the following method, you can create a bean of HandlerMethodArgumentResolver and implement logic to resolve the argument.

@GetMapping("/users")
public Page<User> getUsers(MyObject object) {

Spring Framework already provides a lot of resolvers to handle various parameters such as AuthenticationPrincipal, CSRF, Session, Cookie, MVC Model, and of course Pageable.

 

Pageable Resolver:

Spring Data comes with PageableHandlerMethodArgumentResolver to resolve pageable parameter from the request URL.

If you send a request /users?size=20&page=2, the Pageable object will be injected to the method parameter.

@GetMapping("/users")
public Page<User> getUsers(Pageable pageable) {
    return userRepository.findAllByStatus(Status.ACTIVE, pageable);
}

Customize PageableHandlerMethodArgumentResolver

To customize the Pageable resolver, we need to create a bean of PageableHandlerMethodArgumentResolverCustomizer , which will be applied at SpringDataWebConfiguration#customizePageableResolver before the pageableResolver() is created SpringDataWebConfiguration#pageableResolver.

PageableHandlerMethodArgumentResolverCustomizer is a SAM (single method interface aka FunctionalInterface). 

Setting max page size

@Bean
public PageableHandlerMethodArgumentResolverCustomizer paginationCustomizer() {
    return pageableResolver -> {
        pageableResolver.setMaxPageSize(20); //default is 2000
        pageableResolver.setPageParameterName("pageNumber"); //default is page
        pageableResolver.setSizeParameterName("elementsPerPage"); //default is size
    };
}

Now the url should be /users?elementsPerPage=20&pageNumber=2 instead of /users?size=20&page=2.

If you pass elementsPerPage more than 20, it will be defaulted back to 20.

Which will be helpful to prevent potential attacks trying to issue an OutOfMemoryError.

Protobuf Apache Pulsar with Spring Boot

Protocol buffers(protobuf) are language and platform neutral mechanism to serialize structured data. We define the data structure in protobuf format and generate source code to write and read data to and from a variety of data streams and using a variety of languages.

Protocol buffers currently support generated code in Java, Python and more.

Apache Pulsar is a cloud-native, distributed messaging and streaming platform.

In this blog we are going to use a simple protobuf message and use protobuf-maven-plugin to generate Java source code and use protobuf as message format(schema type) for Apache Pulsar pub-sub application.

1) Protobuf model + Java Source generation:

By default protobuf-maven-plugin looks at src/main/proto folder for the .proto files. We have following proto files to represent Person and Greeting objects

Person.proto

syntax = "proto3";
package app.model;
message Person {
  string fName = 1;
  string lName = 2;
}

Greeting.proto:

syntax = "proto3";
package app.model;
message Greeting {
  string greeting = 1;
}

Here's the basic configuration for protobuf-maven-plugin. It also requires os-maven-plugin. Please refer to the github project for the full source.

 

<plugin>
    <groupId>org.xolstice.maven.plugins</groupId>
    <artifactId>protobuf-maven-plugin</artifactId>
    <version>0.6.1</version>
    <configuration>
        <protocArtifact>com.google.protobuf:protoc:3.12.0:exe:${os.detected.classifier}</protocArtifact>
        <pluginId>grpc-java</pluginId>
        <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.25.0:exe:${os.detected.classifier}</pluginArtifact>
        <clearOutputDirectory>true</clearOutputDirectory>
    </configuration>
    <executions>
        <execution>
            <goals>
                <goal>compile</goal>
            </goals>
        </execution>
    </executions>
</plugin>

 

2) Spring Boot + Apache Pulsar

PulsarClient Bean: Spring Boot doesn't have an official auto configuration for Apache Pulsar yet. So, we have to create the PulsarClient bean ourselves.

 

@Bean
PulsarClient pulsarClient() throws PulsarClientException {
    return PulsarClient.builder()
            .serviceUrl("pulsar://localhost:6650")
            .build();
}

Also the Producer. Please note the parameter of newProducer. We are using PROTOBUF  as the schema type.

Producer bean: We will need to create the producer bean for Person model at main-app and for Greeting model at greeting-service.

@Bean
Producer<THE_MODEL> personProducer(PulsarClient pulsarClient) throws PulsarClientException {
    return pulsarClient.newProducer(Schema.PROTOBUF(THE_MODEL.class))
            .topic(THE_TOPIC)
            .create();
}

 

Pulsar Listener Config: We can register the message listener using @PostConstruct as follows

final PulsarClient pulsarClient;

@PostConstruct
private void initConsumer() throws PulsarClientException {
    pulsarClient
            .newConsumer(Schema.PROTOBUF(THE_MODEL.class))
            .topic(THE_TOPIC)
            .subscriptionName(SUBSCRIPTION_NAME)
            .messageListener((consumer, msg) -> {
                
                //message handler logic

            })
            .subscribe();
}

That's all the configuration we will need.

3. How to run?

1) Clone the project https://github.com/gtiwari333/spring-protobuf-grpc-apache-pulsar and run mvn clean compile to generate java sources and compile the project

2) Run apache pulsar instance using docker(you can run it manually)

docker run -it   -p 6650:6650   -p 8080:8080   apachepulsar/pulsar:2.2.0   bin/pulsar standalone

3) Start GreetingApp and MainApp

4) Hit localhost:8082/greet/Joe/Biden to publish a message. You will see this being received by greeting-service and the greeting being sent back to the queue which will be received by main-app.

The project structure:

├── pom.xml
├── greeting-service
│   ├── pom.xml
│   └── src
│       └── main
│           ├── java
│           │   └── gt
│           │       └── greeting
│           │           └── GreetingApp.java
│           └── resources
│               └── application.properties
├── main-app
│   ├── pom.xml
│   └── src
│       └── main
│           ├── java
│           │   └── gt
│           │       └── mainapp
│           │           └── MainApp.java
│           └── resources
│               └── application.properties
├── protobuf-model
│   ├── pom.xml
│   └── src
│       └── main
│           └── proto
│               ├── Greeting.proto
│               └── Person.proto

 

References:

• https://developers.google.com/protocol-buffers/docs/overview
• https://pulsar.apache.org/docs/en/client-libraries-java/
• GitHub project https://github.com/gtiwari333/spring-protobuf-grpc-apache-pulsar

Read all table and columns in Jpa/Hibernate

How to get metadata about all table and columns managed by JPA/Hibernate?

There are many ways to get a list of table and columns in your project that uses JPA/Hibernate. Each has pros and cons.

Option A) Direct Query on INFORMATION_SCHEMA.

Simplest way is to do direct query on INFORMATION_SCHEMA  or similar schema that the database internally uses.

For MySQL, H2, MariaDB etc the following would work. We will need specific query for each database.

SELECT * from INFORMATION_SCHEMA.TABLES
SELECT * FROM INFORMATION_SCHEMA.COLUMNS

Option B) DB Independent query using JDBC API

We can do DB independent query by using JDBC API to return the Metadata. It would use DB specific query provided by DB driver to return the metadata.

DataSource ds = ; //create/wire DataSource object
DatabaseMetaData metaData = ds.getConnection().getMetaData();
ResultSet schemasRS = metaData.getSchemas();
ResultSet tablesRS = metaData.getTables(null, null, null, new String[]{"TABLE"});

We can iterate over the ResultSet to get the schema, table and columns. It would return everything that the Database has.

Option C) Use EntityManager MetaModel to read Entity classes

In order to retrieve only the entity/tables that the application uses, we can rely on some Java Reflection magic as following:

EntityManager em; //autowire the bean
MetamodelImplementor metaModelImpl = (MetamodelImplementor) em.getMetamodel();
List<String> tableNames = metaModelImpl
    .entityPersisters()
    .values().stream()
    .map(ep -> ((AbstractEntityPersister) ep).getTableName())
    .toList();

Option D) Use Hibernate Magic

Use hibernate's Metadata class that stores the ORM model determined by provided entity mappings.

org.hibernate.boot.Metadata metadata; //getting the Metadata is tricky though

for (PersistentClass persistentClass : metadata.getEntityBindings()) {
    tableNames.add(persistentClass.getTable().getExportIdentifier());
}