Spring data Pagination - set max page size and other customizations

Background:

HandlerMethodArgumentResolver is a strategy interface to resolve method parameters in context of given context. So, if you want to automatically resolve the parameter MyObject in the following method, you can create a bean of HandlerMethodArgumentResolver and implement logic to resolve the argument.

@GetMapping("/users")
public Page<User> getUsers(MyObject object) {

Spring Framework already provides a lot of resolvers to handle various parameters such as AuthenticationPrincipal, CSRF, Session, Cookie, MVC Model, and of course Pageable.

 

Pageable Resolver:

Spring Data comes with PageableHandlerMethodArgumentResolver to resolve pageable parameter from the request URL.

If you send a request /users?size=20&page=2, the Pageable object will be injected to the method parameter.

@GetMapping("/users")
public Page<User> getUsers(Pageable pageable) {
    return userRepository.findAllByStatus(Status.ACTIVE, pageable);
}

Customize PageableHandlerMethodArgumentResolver

To customize the Pageable resolver, we need to create a bean of PageableHandlerMethodArgumentResolverCustomizer , which will be applied at SpringDataWebConfiguration#customizePageableResolver before the pageableResolver() is created SpringDataWebConfiguration#pageableResolver.

PageableHandlerMethodArgumentResolverCustomizer is a SAM (single method interface aka FunctionalInterface). 

Setting max page size

@Bean
public PageableHandlerMethodArgumentResolverCustomizer paginationCustomizer() {
    return pageableResolver -> {
        pageableResolver.setMaxPageSize(20); //default is 2000
        pageableResolver.setPageParameterName("pageNumber"); //default is page
        pageableResolver.setSizeParameterName("elementsPerPage"); //default is size
    };
}

Now the url should be /users?elementsPerPage=20&pageNumber=2 instead of /users?size=20&page=2.

If you pass elementsPerPage more than 20, it will be defaulted back to 20.

Which will be helpful to prevent potential attacks trying to issue an OutOfMemoryError.

Protobuf Apache Pulsar with Spring Boot

Protocol buffers(protobuf) are language and platform neutral mechanism to serialize structured data. We define the data structure in protobuf format and generate source code to write and read data to and from a variety of data streams and using a variety of languages.

Protocol buffers currently support generated code in Java, Python and more.

Apache Pulsar is a cloud-native, distributed messaging and streaming platform.

In this blog we are going to use a simple protobuf message and use protobuf-maven-plugin to generate Java source code and use protobuf as message format(schema type) for Apache Pulsar pub-sub application.

1) Protobuf model + Java Source generation:

By default protobuf-maven-plugin looks at src/main/proto folder for the .proto files. We have following proto files to represent Person and Greeting objects

Person.proto

syntax = "proto3";
package app.model;
message Person {
  string fName = 1;
  string lName = 2;
}

Greeting.proto:

syntax = "proto3";
package app.model;
message Greeting {
  string greeting = 1;
}

Here's the basic configuration for protobuf-maven-plugin. It also requires os-maven-plugin. Please refer to the github project for the full source.

 

<plugin>
    <groupId>org.xolstice.maven.plugins</groupId>
    <artifactId>protobuf-maven-plugin</artifactId>
    <version>0.6.1</version>
    <configuration>
        <protocArtifact>com.google.protobuf:protoc:3.12.0:exe:${os.detected.classifier}</protocArtifact>
        <pluginId>grpc-java</pluginId>
        <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.25.0:exe:${os.detected.classifier}</pluginArtifact>
        <clearOutputDirectory>true</clearOutputDirectory>
    </configuration>
    <executions>
        <execution>
            <goals>
                <goal>compile</goal>
            </goals>
        </execution>
    </executions>
</plugin>

 

2) Spring Boot + Apache Pulsar

PulsarClient Bean: Spring Boot doesn't have an official auto configuration for Apache Pulsar yet. So, we have to create the PulsarClient bean ourselves.

 

@Bean
PulsarClient pulsarClient() throws PulsarClientException {
    return PulsarClient.builder()
            .serviceUrl("pulsar://localhost:6650")
            .build();
}

Also the Producer. Please note the parameter of newProducer. We are using PROTOBUF  as the schema type.

Producer bean: We will need to create the producer bean for Person model at main-app and for Greeting model at greeting-service.

@Bean
Producer<THE_MODEL> personProducer(PulsarClient pulsarClient) throws PulsarClientException {
    return pulsarClient.newProducer(Schema.PROTOBUF(THE_MODEL.class))
            .topic(THE_TOPIC)
            .create();
}

 

Pulsar Listener Config: We can register the message listener using @PostConstruct as follows

final PulsarClient pulsarClient;

@PostConstruct
private void initConsumer() throws PulsarClientException {
    pulsarClient
            .newConsumer(Schema.PROTOBUF(THE_MODEL.class))
            .topic(THE_TOPIC)
            .subscriptionName(SUBSCRIPTION_NAME)
            .messageListener((consumer, msg) -> {
                
                //message handler logic

            })
            .subscribe();
}

That's all the configuration we will need.

3. How to run?

1) Clone the project https://github.com/gtiwari333/spring-protobuf-grpc-apache-pulsar and run mvn clean compile to generate java sources and compile the project

2) Run apache pulsar instance using docker(you can run it manually)

docker run -it   -p 6650:6650   -p 8080:8080   apachepulsar/pulsar:2.2.0   bin/pulsar standalone

3) Start GreetingApp and MainApp

4) Hit localhost:8082/greet/Joe/Biden to publish a message. You will see this being received by greeting-service and the greeting being sent back to the queue which will be received by main-app.

The project structure:

├── pom.xml
├── greeting-service
│   ├── pom.xml
│   └── src
│       └── main
│           ├── java
│           │   └── gt
│           │       └── greeting
│           │           └── GreetingApp.java
│           └── resources
│               └── application.properties
├── main-app
│   ├── pom.xml
│   └── src
│       └── main
│           ├── java
│           │   └── gt
│           │       └── mainapp
│           │           └── MainApp.java
│           └── resources
│               └── application.properties
├── protobuf-model
│   ├── pom.xml
│   └── src
│       └── main
│           └── proto
│               ├── Greeting.proto
│               └── Person.proto

 

References:

• https://developers.google.com/protocol-buffers/docs/overview
• https://pulsar.apache.org/docs/en/client-libraries-java/
• GitHub project https://github.com/gtiwari333/spring-protobuf-grpc-apache-pulsar