GIT write useful commit messages

What's harder than choosing a variable name? IMO its composing a perfect code comment and commit message. Today we will discuss some bad commit messages, why they are bad and what we can do to improve those?

You made all the code updates, wrote tests and now its time to commit them with a commit message. Is "Code Updated" a sufficient? What about "Issue fixed"? Just imagine yourself reading the commit message some time later. Just like code comment, commit message should add context about why the code update was necessary? 

A good commit message helps a future developer(it might be you) to reestablish the context around a piece of code whenever the developer needs to dig through the history to find out why the code was changed. It also helps the code reviewer to what to expect on the code.

Unless your team follows squash merge (where all the individual commits in the PR are combined and the person merging the PR can put a single/meaningful commit message) you should pay extra attention when choosing. Commit messages like the following add no value. A good developer should never write these kind of commit messages.

Bad commit messages(taken from real life examples):

•  issue fix
•  more changes
  
•  logic updated
•  updated code again
•  added import function
•  updated dialog box UI
•  deleted column
  

Good practices:

Describe why the code was updated?

"Issue fix" doesn't indicate which issue number and what was the cause of issue? "Logic updated" is super obvious and serves no purpose. "updated dialog box UI" doesn't indicate which dialog box and why it was updated. "added import function" doesn't indicate which data type does it import?

Refer the Issue/Jira/Story number

All the code changes should be followed by a Jira/Story created by someone. Noone should just go ahead and make a arbitrary code change. Most of the git providers (github, bitbucket, gitlab etc) support hyperlinking of Jira/Issue number from commit message. eg: UI_APP:4555 Fixed broken button on IE9 caused by unsupported JS function

Commit frequency: per logical change

Its good to do a new commit per logical change. eg: do a commit once the functional piece is working:

• new column 'purchase_amount' added on table
  
• updated the backend to save new purchase_amount column
• updated UI to display new column

For bigger update, use a heading and bullet points. The header can be short.

Commit frequency: commit often to have a periodic checkpoint

Imagine you are working on a BIG feature where you modified several files. At some point you are able to make few things working (point X). But when you work on additional changes the things that worked earlier started breaking. Later you are in a position where you don't know how to revert the new changes to get back to the point X where few things were function. If you had committed your changes at point X, it would have made it super easy to revert local changes to that point.

Few good examples:   

• dialog box UI updated to support multiple callback functions on both Cancel and Save event –> instead of updated dialog box ui     
• removed the employer contribution period field as we're no longer using it –> instead of removed employer contribution period field     
• add member import function for the employer user –> instead of added import function    

 

 

 

KeyCloak docker-compose import realm file

Example of docker-compose for keycloak 18 with ability to import a realm file:

It assumes the realm-export.json file is in /PATH_OF_realm-export.json_FILE/

docker-compose.yml:

version: '3'
services:
    keycloak:
        image: 'quay.io/keycloak/keycloak:18.0.2'
        container_name: keycloak
        command: [ 'start-dev --import-realm' ]
        environment:
            - KEYCLOAK_DB=dev-file
            - KEYCLOAK_ADMIN=admin
            - KEYCLOAK_ADMIN_PASSWORD=admin
            - KEYCLOAK_FEATURES=scripts
            - KEYCLOAK_HTTP_PORT=8080
        volumes:
            - /PATH_OF_realm-export.json_FILE/:/opt/keycloak/data/import
        ports:
            - 8082:8080

Kafka - a working docker-compose

Here's a working docker-compose that runs zookeeper, kafka and kafka-ui in same network.

version: '3.8'
services:

    zookeeper:
        container_name: zookeeper
        image: confluentinc/cp-zookeeper:latest
        hostname: zookeeper
        restart: always
        environment:
            ZOOKEEPER_SERVER_ID: 1
            ZOOKEEPER_CLIENT_PORT: 2181
            ZOOKEEPER_TICK_TIME: 2000
            ZOOKEEPER_INIT_LIMIT: 5
            ZOOKEEPER_SYNC_LIMIT: 2
        ports:
            - 2181:2181

    kafka:
        container_name: kafka
        image: confluentinc/cp-kafka:latest
        depends_on:
            - zookeeper
        restart: always
        environment:
            KAFKA_BROKER_ID: 1
            KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
            KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
            KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092,RMOFF_DOCKER_HACK://kafka:9093
            KAFKA_LISTENERS: PLAINTEXT://:9092,RMOFF_DOCKER_HACK://:9093
            KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,RMOFF_DOCKER_HACK:PLAINTEXT
            KAFKA_INTER_BROKER_LISTENER_NAME: RMOFF_DOCKER_HACK
        ports:
            - 9092:9092

    kouncil:
        container_name: kouncil
        image: consdata/kouncil:latest
        restart: always
        depends_on:
            - kafka
        environment:
            bootstrapServers: kafka:9093
            BOOTSTRAPSERVERS: kafka:9093
            BOOTSTRAP_SERVERS: kafka:9093
        ports:
            - 8888:8080

 

Spring Boot - shutdown and restart application programmatically

Today we will be discussing how to shutdown and restart a Spring application programmatically.

1) New Project

First let's create a simple spring boot app with web dependency so that we can setup an endpoint to trigger the shutdown and restart. Let's head over to  https://start.spring.io/ and choose web and other dependencies that you need

2) Few words about ApplicationContext and ConfigurableApplicationContext

ApplicationContext is the central interface that represents the Spring IoC container and is responsible for instantiating, configuring, and assembling the beans.  ConfigurableApplicationContext is another interface that extends  ApplicationContext and provides facilities to configure refresh, startup & shutdown of the application and allows registration of custom listeners to respond to those events.

3) Let's code!

3.1) Simple App

Run the application and store ConfigurableApplicationContext in a static variable appCtx so that we can use it later

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ConfigurableApplicationContext;

@SpringBootApplication
public class App {

    static ConfigurableApplicationContext appCtx;

    public static void main(String[] args) {
        appCtx = SpringApplication.run(App.class, args);
    }

3.2) Shutdown - call ConfigurableApplicationContext.close() 

This closes the application context, releasing all resources and locks that the implementation might hold. This includes destroying all cached singleton beans.

static void shutdown() {
    appCtx.close();
}

3.3) Restart

We know how to stop and start the application. To stop, we can simply do:

appCtx.close();

And to start, we can simply call the SpringApplication.run(..)

appCtx = SpringApplication.run(App.class, args);

But the catch here is, appCtx.close() would exit the application and the SpringApplication.run(..) won't be able to run

So, a solution would be to use Daemon Thread to shutdown and start the application. Daemon thread is a type of thread that can run independently in the background(independent of the main thread). 

Thread t = new Thread(() -> {
    appCtx.close();
    appCtx = SpringApplication.run(...);
});

t.setDaemon(false);
t.start();

3.4) Passing the original program arguments

We may want to use the original arguments that were passed to run method when we restart the application.

 SpringApplication.run(App.class, args);

We can do that by either storing String[] args in a static variable or use ApplicationArguments bean that Spring Boot creates to store the arguments.

var args = appCtx.getBean(ApplicationArguments.class);      //retrieve
appCtx = SpringApplication.run(App.class, args.getSourceArgs());  //pass

3.5) Web endpoints to trigger the shutdown and restart

@RestController
class Ctrl {

    @GetMapping("/shutdown")
    void shutdown() {
        App.shutdown();
    }

    @GetMapping("/restart")
    void restart() {
        App.restart();
    }
}

4) Complete Example:

package gt.restart;

import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@SpringBootApplication
public class App {

    static ConfigurableApplicationContext appCtx;

    public static void main(String[] args) {
        appCtx = SpringApplication.run(App.class, args);
    }

    static void shutdown() {
        if (appCtx.isActive()) {
            appCtx.close();
        }
    }

    static void restart() {
        var args = appCtx.getBean(ApplicationArguments.class);

        Thread thread = new Thread(() -> {
            appCtx.close();
            appCtx = SpringApplication.run(App.class, args.getSourceArgs());
        });

        thread.setDaemon(false);
        thread.start();
    }

}

@RestController
class Ctrl {

    @GetMapping("/shutdown")
    void shutdown() {
        App.shutdown();
    }

    @GetMapping("/restart")
    void restart() {
        App.restart();
    }
}

5) Test

Call the endpoints

GET http://localhost:8080/restart

GET http://localhost:8080/shutdown

Read Request parameter String from URL in Spring/HttpServlet

Read entire query string(request parameter) in Spring Boot/HttpServlet

In Spring Boot and other Java web frameworks, we capture the request parameters (aka query parameters) and map them to individual variable or a Map<String,String>. 

But, if you want to read the entire query parameter in its raw form ( eg: to read name=Ganesh&country=Nepal&ageLt=30&page=1 from request: http://localhost:8080/search?name=Ganesh&country=Nepal&ageLt=30&page=1), you can utilize HttpServletRequest.getQueryString()

Example:

@GetMapping("/test1")
void endpoint1(HttpServletRequest req) {
    var qs = req.getQueryString()  //returns the entire string
    qs.split("&") //split to get the individual parameters
}

Note: Spring Boot automatically resolves the HttpServletRequest object ( its the real holder of request object in Http Servlet environment)

GIT delete merged branches

How to delete merged branches from local copy:

When you work on a project for a while, there is a good chance that you will have a list of old/already merged branches showing on your local git. 

When you run $git branch it will show all the branches since you started working on the project including the ones that are not currently active.

$git branch
    PROJX-1024-add-new-button
    PROJX-1026-add-another-button
    PROJX-1027-titleupdates
    build-fix
    develop
    release
    master 

Here's how you can find the merged branches: It will only list a branch if it exists in local but not in remote(deleted from remote after merging).

$git branch --merged
    PROJX-1024-add-new-button
    build-fix
    develop
    release

Here's how you delete the merged branches except develop and release

$git branch --merged | egrep -v "(^\*|develop|release)" | xargs git branch -d

You can add any number of branches to the skip list separated by pipe |

$git branch --merged | egrep -v "(^\*|develop|release|another_branch|yet_another_branch)" | xargs git branch -d

Java view,download,filter email using javax.mail

Java SDK doesn't come with inbuilt mail API but Java-EE (now called Jakarta EE) provides several utility library like json, servlet, web-service, jms, validation, xml, regexp including email.

In this example I'm going to show how we can connect to email server (eg: gmail) and download/view seen/unseen emails.

Required Dependency :jakarta.mail

Add this dependency with latest version in your pom.xml or manually download/add the jar file to your project's lib.

<dependency>
    <groupId>com.sun.mail</groupId>
    <artifactId>jakarta.mail</artifactId>
</dependency>

The API Usage:

Ideally, the steps would be as following:

- Create Session with host, protocal type etc

- Get message store (Store) for the email protocol. The supported protocols are IMAP, POP, SMTP etc

- Connect to the host using username/password

- Open folder and search for messages

The API Usage with code example:

- Create Session with host, protocol type etc

Properties properties = new Properties();
properties.put("mail.imap.host", host);
properties.put("mail.store.protocol", protocal);
Session emailSession = Session.getDefaultInstance(properties);

 - Get message store (Store) for the email protocol. The supported protocols are IMAP, POP, SMTP etc

Store store = emailSession.getStore(protocal);

- Connect to the host using username/password

store.connect(host, user, password);

- Open folder and search for messages

Folder emailFolder = store.getFolder("INBOX");
emailFolder.open(Folder.READ_ONLY);

Search using filters

Flags seen = new Flags(Flags.Flag.SEEN);
FlagTerm unseenFlagTerm = new FlagTerm(seen, false);
Message[] messages = emailFolder.search(unseenFlagTerm);

Complete Example:

import javax.mail.*;
import javax.mail.search.FlagTerm;
import java.util.Properties;

public class MailllllIMAP {

    public static void check(String host, String protocal, String user, String password) {
        try {
            Properties properties = new Properties();
            properties.put("mail.imap.host", host);
            properties.put("mail.store.protocol", protocal);
            Session emailSession = Session.getDefaultInstance(properties);

            Store store = emailSession.getStore(storeType);
            store.connect(host, user, password);
            Folder emailFolder = store.getFolder("INBOX");
            emailFolder.open(Folder.READ_ONLY);

            //unseen email filter
            Flags seen = new Flags(Flags.Flag.SEEN);
            FlagTerm unseenFlagTerm = new FlagTerm(seen, false);
            Message[] messages = emailFolder.search(unseenFlagTerm);

            System.out.println("Total messages: " + messages.length);

            for (Message m : messages) {
                System.out.println(("From: " + m.getFrom()[0]));
                System.out.println(("Subject: " + m.getSubject()));
                System.out.println(("Subject: " + m.getContent()));
                //other fields
            }


            //close or do more stuff
            emailFolder.close(false);
            store.close();

        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public static void main(String[] args) {

        String host = "pop.gmail.com";
        String protocal = "imaps";
        String username = "XXXX@gmail.com";
        String password = "XXXXXXX";

        check(host, protocal, username, password);

    }

}

What if you get an error like this?

    AuthenticationFailedException: [AUTH] Username and password not accepted.

You get this error when trying to gmail. You will need to turn ON the "Allow less secure apps" option on https://www.google.com/settings/security/lesssecureapps

Java HttpClient tutorial with File Upload example

Java 11 comes with a nice HttpClient API/Implementation so that we no longer need to rely on external libraries like Apache HttpClient execute http requests.

It has the following simple classes(namely HttpClient, HttpRequest, HttpResponse, BodyPublisher, BodyHandler) and they all follow builder pattern to create objects.

Creating HttpClient:

HttpClient client = HttpClient.newBuilder().build();

Creating request:

HttpRequest request = HttpRequest.newBuilder()
        .header("key1", "value1")
        .header("key2", "value2")
        .uri(URI.create("http://localhost:8080/hello"))
        .POST(HttpRequest.BodyPublishers.ofString("Request Body"))
        .build();

Sending request:

client.send(request, HttpResponse.BodyHandlers.APPROPRIATE_HANDLER);

Here, BodyPublisher and BodyHandler can be used to create request body and process returned response respectively. They both support string, byte, file, input stream etc

Upload file using BodyPublishers.ofFile

Normally file upload using a web page are implemented using multi-part(file data with additional properties). But in this example we are going to upload file's byte array in request body and file name in request param.

import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.file.Path;

public class HttpTest {
  public static void main(String[] args) throws Exception {
    uploadFile(Path.of("testFile.txt"));
  }

  public static void uploadFile(Path file) throws IOException, InterruptedException {
    HttpClient client = HttpClient.newBuilder().build();

    HttpRequest request = HttpRequest.newBuilder()
      .uri(URI.create("http://localhost:8080/uploadFile?uploader=HttpTestApp&fileName=" 
                  + file.getFileName()))
      .POST(HttpRequest.BodyPublishers.ofFile(file))
      .build();

    client.send(request, HttpResponse.BodyHandlers.ofString());
  }
}

To make the example complete, let's assume we have a following simple Spring Boot app that receives the file byte array as request body:

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.*;

import java.io.*;

@SpringBootApplication
public class FileUploadApp {
    public static void main(String[] args) {
        SpringApplication.run(FileUploadApp.class, args);
    }
}

@Controller
class FileUplCtrl {

  @PostMapping(path = "/uploadFile")
  public void addPhoto(@RequestBody byte[] barr,
              @RequestParam(name = "uploader") String uploader,
              @RequestParam(name = "fileName") String fileName) throws Exception {

   System.out.println("Received file " + fileName + " , uploaded by " + uploader + " Size: " + barr.length);

    //write to disk
    try (OutputStream os = new FileOutputStream(new File("UPL" + fileName))) {
         os.write(barr);
    }

  }
}

Required Dependency on Spring Boot app:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>

Enjoy coding!

Thymeleaf re-usable pagination component- Spring MVC/Bootstrap

In this blog, we will discuss how we can create a reusable pagination component using Thymeleaf and Spring MVC/Data.

Goal is to create a pagination UI similar to this:

 

Controller 

Spring resolves the size and page parameter from URL : /users?size=20&page=1 to Pageable object at the Controller methods. We are returning Page<> instead of simple list. Page contains all the information that we need to create the pagination UI as above.

@GetMapping("/articles")
public String userHome(Model model, Pageable pageable) {
    Page<Article> page= articleService.findAllByStatus(Status.PUBLISHED, pageable);
    model.addAttribute("articles", page);
    return "home-page";
}

@GetMapping("/users")
public String userHome(Model model, Pageable pageable) {
    Page<User> page= userRepo.findAllByStatus(Status.ACTIVE, pageable);
    model.addAttribute("users", page);
    return "user-list";
}

 

Pagination at home-page.html   

<th:block th:each="article : ${articles}">
   <!-- display article in a table row or card/box -->
</th:block>

<!--pager at bottom -->
<th:block th:replace="_fragments/_utils :: pagination(${articles})">
</th:block>

Pagination at user-list.html

<th:block th:each="user : ${users}">
   <!-- display user in a table row or card/box -->
</th:block>
<!--pager at bottom -->
<th:block th:replace="_fragments/_utils :: pagination(${users})">
</th:block>

Reusable Fragment Definition at _fragments/_utils.html

Note that the pages are 0 based meaning page=0 equals the first page of data. You can make it to 1 based using the steps described here: https://ganeshtiwaridotcomdotnp.blogspot.com/2020/09/spring-pageable-start-index-with-one.html. You might also need to update the indices below to to handle that.

<th:block th:fragment="pagination(pagedObject)">

    <!-- works with org.springframework.data.domain.Page<pagedObject>, -->

    <div th:if="${pagedObject.getTotalPages() != 1}" class="form-group col-md-11 pagination-centered">
        <ul class="pagination">
            <!-- page number start with 0, totalPages returns actual number of pages -->
            <li th:classappend="${pagedObject.getNumber() == 0} ? disabled" class="page-item">
                <a class="page-link" th:href="@{''(size=${pagedObject.getSize()}, page=0)}">&laquo;</a>
            </li>
            <li th:classappend="${pagedObject.getNumber() == 0} ? disabled" class="page-item">
                <a class="page-link"
                   th:href="@{''(size=${pagedObject.getSize()}, page=${pagedObject.getNumber() -1})}">&larr;</a>
            </li>
            <li th:classappend="${pagedObject.getNumber() == (page  )} ? 'active pointer-disabled'"
                th:each="page : ${#numbers.sequence(0, pagedObject.getTotalPages() -1)}" class="page-item">
                <a class="page-link" th:href="@{''(size=${pagedObject.getSize()}, page=${page})}"
                   th:text="${page + 1}"></a>
            </li>
            <li th:classappend="${pagedObject.getNumber() + 1 == pagedObject.getTotalPages()} ? disabled"
                class="page-item">
                <a class="page-link"
                   th:href="@{''(size=${pagedObject.getSize()}, page=${pagedObject.getNumber() + 1})}">&rarr;</a>
            </li>
            <li th:classappend="${pagedObject.getNumber() + 1 == pagedObject.getTotalPages()} ? disabled"
                class="page-item">
                <a class="page-link"
                   th:href="@{''(size=${pagedObject.getSize()}, page=${pagedObject.getTotalPages()})}">&raquo;</a>
            </li>
        </ul>

        <p class="text-muted small">
            Showing page <span th:text="${pagedObject.getNumber() +1}"></span> of <span
            th:text="${pagedObject.getTotalPages()}"></span>.
            Total: <span th:text="${pagedObject.getTotalElements()}"></span>.

        </p>
    </div>


    <div th:if="${pagedObject.getTotalElements() ==0}">
        <div class="alert alert-warning" role="alert">
            <span>No items available.</span>
        </div>
    </div>


</th:block>

Reference: https://github.com/gtiwari333/spring-boot-web-application-seed/blob/master/main-app/src/main/resources/templates/_fragments/_utils.html

Thymeleaf pass parameter to fragment

Making reusable fragment/UI Components in Thymeleaf

Thymeleaf supports creation of 'function like' mechanism and pass parameters to make the UI components reusable.

In this example we are going to create a reusable alert component using thymeleaf:

The fragment definition is similar to how a function/method is defined:

File: _fragments/_utils.html

<div th:fragment="alertBox(message,type)">
    //do stuff with message and type
</div>

Let's call with some parameters in another file:

<div th:replace="_fragments/_utils::alertBox ('A message','success')">
</div>

The parameter order is not important if you do like this:

<div th:replace="_fragments/_utils::alertBox (type = 'success', message='A message' )">
</div>

A complete example:

<div th:fragment="alertBox (message,type)">
    <div
        th:classappend="|
            ${type == 'success' ? 'alert-success': ''}
            ${type == 'info' ? 'alert-info': ''}
            ${type == 'danger' ? 'alert-danger': ''} |"
        class="alert alert-dismissible">
        <a href="#" class="close" data-dismiss="alert" aria-label="close">&times;</a>
        <span th:text="${message}"></span>
    </div>
</div>

Calling it:

<div th:replace="_fragments/_utils :: alertBox ('Hello this is a success!','success')">
</div>

<div th:replace="_fragments/_utils :: alertBox ('Danger! danger! .......','danger')">
</div>


<div th:replace="_fragments/_utils :: alertBox ( type = 'success', message='Success message' )">
</div>

Output: