Say hi to Spring Boot!

I am here after a long time to pen down some thoughts for just getting back to the habit of writing. I hope I’ll be able to continue sharing my thoughts more frequently.

Since its been a while, I thought I’ll share something that is not too complicated as it might make it tough for me to get out of my inertia. So, I am going to share an opinionated way of bootstrapping a light-weight java web-application.

With new languages and features coming out for public use every now and then, it is high time that legacy languages such as Java (I may no longer be able to claim this, as Java is quickly catching up to the latests trends with more sexy language features with the new java release cycle) catch up to speed with cool features other languages/frameworks offer. There were times, when I used to spend a whole day just to kick-start a basic skeleton web app with a ton of configuration files and downloading a cryptic application server and copying my class files to weird locations. But with a lot of frameworks that support java integration, things have gotten much better.

There’s plethora of easy-to-use frameworks that has made Java backend/full-stack engineer’s life a lot better. But I’d like to share my experience on how easy it is to setup a skeleton web-app using my favorite Springboot (Although I use dropwizard for my day-to-day activities at my work for various reasons, Spring has always been my favorite framework of choice ever since my revelation about things getting easier in Java).

I can keep blabbering about Spring framework forever. But I prefer to see things in action. So let me start with my rambling!

Say hi to Initializr!

The biggest pain for getting started with any framework is actually “getting started“! With spring boot, getting started is as easy as clicking a few buttons. Don’t believe me, let me introduce you to spring initializr.

Spring initializr page

For setting up a spring-boot project using spring initializr, you need to;

1. Setup your build tool (We are using maven as part of this post) as described here (Ignore if you already have maven installed)
2. Go to https://start.spring.io/
3. Fill a form to enter metadata about your project.
4. Decide on dependencies to add to your project (I did not add any for this demo).
5. Click on generate!

Well, once we extract the zip file, we need to open it using any IDE (I am using IntelliJ here but I wouldn’t expect any issues with Eclipse, Netbeans, etc). Here is how your extracted project would look like.

Extracted project structure

You can find this as part of my commit here in github if you are following along with me. All you now need to do is to build the project and run the DemoApplication.java file. If you like to run it from terminal like I prefer, run the following command from your root directory.

mvn spring-boot:run

Run using maven

If you observe now, the app starts and shuts down immediately, let’s fix that next.

But where’s the web app?

Like I said, since we did not choose any dependency in the spring initializr page. Spring initializr assumed that we were not building a web-app and gave us the bare minimum configuration for running our app.

There is a reason why I did not choose any dependency when I bootstrapped the app. Often, when we start with an app, we do not know the exact set of dependencies we’d need to start off with. It is crucial for us to know how to add additional dependencies after we started working on the app. Easy way would be to click on the Explore button on the spring initializer web-page by choosing your dependency. Let me show how easy it is in the following gif.

Copying the web starter dependency

Now that you’ve copied the required dependency to power a web app, let’s go ahead and change our pom.xml so we could pull in the needed dependencies for bootstrapping a web-app.

Your pom file should look like my pom file in this commit.

Great! now we need to just add some magic and we are all set. For every web-app (Restful web app), you need the following logical entities for defining your restful web service;

1. HTTP method (Defines the type of operation)
2. Endpoint name (Defines a name for the operation)
3. Parameters & Request payload (Defines user input for processing)
4. Response (Defines how the response should look like)

These four things are what we are going to define in our DemoApplication.java.

Your DemoApplication.java should look like the one I have in this commit here.

That’s it! we are done with the code changes. Build your project and run it using the same command mvn spring-boot:run and now you should logs similar to the below screenshot.

Lets now open a browser and navigate to the following url

http://localhost:8080/sayHi?name=springboot

This should print the text Hi from springboot in your browser.

Congratulations your spring-boot app is ready and running with absolutely no configuration and no external server setup.

I know this is a lot of magic, but we’ve barely scratched the surface. There is a LOT of magic going on behind the scenes. I will attempt my take on it as part of a separate post in future. So long!

Integration graph when using Spring Integration Java DSL

The drawing shows me at a glance what would be spread over ten pages in a book. –Ivan Turgenev

In my recent post on Spring Integration: A hands on introduction, I have a diagram that illustrates a spring-integration flow. I feel that such an illustration, gives us an awesome way to both document your flow and act as a dashboard to look thru when the spring-integration project has been deployed to have near real-time stats on what is the load on a channel, speed thru which your messages are flowing thru, etc.

If you have used spring-integration using XML config, the IDE‘s would have this straight out of the box (Both in STS Eclipse & IntelliJ Idea).

Integration flow graphs for spring-integration project in Eclipse and IntelliJ IDEA IDE‘s

Well, if you have gone thru that post, you would have noticed that we did not use XML configs for the project, instead, we used the new and cool Java DSL to define out flow. One downside of this I feel to this approach is that, since this whole Java DSL way of defining the flow is fairly new and since our flows get generated dynamically when spring builds up its context, we have no plugins for any IDE‘s (At-least, none to my knowledge) that will give us this integration-graph out of the box.

Fortunately, spring-integration folks have not taken this lightly, we do have a way to visualize the flows when using spring-integration with Java DSL. Its not too straight-forward, but its not difficult either. Let’s today see how to visualize your integration-flows using the project, Spring-Flo.

The link for that project will take you to the angular-1.x branch of the project. That is because, currently the whole project is in the process of migration to angular 4/5. Hence the stable version that we can use in the meantime is the one that is in the angular-1.x branch. I will update this post if I find a newer stable version.

So, that being said, we need to use this external project in-order to visualize your flow, which is not too bad I feel (IMO, it is better than not having any means to visualize your flow). Also, I feel it would be giving more cleaner way to have a dashboard kind of view to see how are things going in real-time. So, I like the idea of having this as a different project.

Let’s right away jump into how to configure your spring-integration project to use spring-flo project. To begin with, this is how spring-flo project will be able to generate the integration-graph for you.

Your integration project will expose an endpoint (/integration) which will give the near real-time stats of your integration components (Channels, Service Activators, etc.). This is just a plain JSON response. The interesting part is that, spring-flo project will latch on to this endpoint, and will provide you with an interesting visualization of the JSON reply.

Dependencies needed:

1. Make the project a web-based one, by including the spring-boot-starter-web. This will give you access to the /integration endpoint when your project boots up.
2. Include the spring-integration-http dependency to use the @EnableIntegrationGraphController on your application class.

	<!– For exposing the /integration endpoint –>
	<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-web</artifactId>
	</dependency>
	<!– For @EnableIntegrationGraphController annotation –>
	<dependency>
	<groupId>org.springframework.integration</groupId>
	<artifactId>spring-integration-http</artifactId>
	</dependency>

view raw

pom.xml

hosted with ❤ by GitHub

Code configuration for your spring-integration project:

1. Add the @EnableIntegrationGraphController to your application class.
2. You need to allow access for spring-flo application to access the endpoint /integration by specifying the allowedOrigins on your annotation.

	@SpringBootApplication
	@EnableIntegrationGraphController(allowedOrigins = "http://localhost:8082")
	public class Application {
	public static void main(String[] args) {
	SpringApplication.run(Application.class, args);
	}
	}

view raw

Application.java

hosted with ❤ by GitHub

Here in this example, I am planning to run the spring-flo application on the same machine in the port 8082, so I’m just specifying that when I’m configuring my integration project.

That’s all you have to do to your spring-integration project.

Running spring-flo project:

1. Clone the spring-flo project to your machine.
2. Checkout to the angular-1.x branch
3. Start your spring-integration project.
4. Run the spring-flo/samples/spring-flo-si project on 8082 since our spring-integration project will be expecting spring-flo to be running on port 8082.
5. Go to localhost:/8082 and click on Load button to load your integration flow graph.

It is not pretty straight-forward, but still, we have something to work on until spring team would give a better solution in the future which I’m pretty confident they would. Until then, we have this web application that provides a wonderful way to watch our integration-flows.

How to get rid of AmazonClientException when using Spring Cloud AWS with spring-boot

I was recently trying to list number of files from S3, and read those files one by one in a service. For doing this I was using spring-boot 2.0.0.RELEASE, spring-cloud-starter-aws (Edgware.SR2) library for auto-configuring my credentials.

I configured my credentials by following the official documentation. You can give that a read in your free time. TLDR; all we have to do if we are using spring-boot to auto-configure your Amazon S3 client is to populate the following properties in your applications.properties or application.yaml file.

cloud.aws.region.static
cloud.aws.credentials.secretKey
cloud.aws.credentials.accessKey

Once you configure these properties, spring-boot should auto-configure aws client for you. But this did not happen for me. When I have these properties filled out, and run the project, I got the following exception.

Caused by: java.lang.NoClassDefFoundError: com/amazonaws/AmazonClientException
	at org.springframework.cloud.aws.context.annotation.OnAwsCloudEnvironmentCondition.matches(OnAwsCloudEnvironmentCondition.java:37) ~[spring-cloud-aws-context-1.2.2.RELEASE.jar:1.2.2.RELEASE]
	at org.springframework.context.annotation.ConditionEvaluator.shouldSkip(ConditionEvaluator.java:109) ~[spring-context-5.0.4.RELEASE.jar:5.0.4.RELEASE]
	at org.springframework.context.annotation.ConfigurationClassParser.processConfigurationClass(ConfigurationClassParser.java:217) ~[spring-context-5.0.4.RELEASE.jar:5.0.4.RELEASE]
	at org.springframework.context.annotation.ConfigurationClassParser.processImports(ConfigurationClassParser.java:606) ~[spring-context-5.0.4.RELEASE.jar:5.0.4.RELEASE]
	... 14 common frames omitted
Caused by: java.lang.ClassNotFoundException: com.amazonaws.AmazonClientException
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[na:1.8.0_152]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_152]
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338) ~[na:1.8.0_152]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_152]
	... 18 common frames omitted

For some reason, spring-boot latest version does not go hand in hand with spring-cloud-dependencies. I tried downgrading spring-boot version and use 1.5.9.RELEASE but then didn’t end up solving the problem. I found out that the problem was that the amazon-java-aws-core library had an upgrade and spring does not handle that well enough, which causes the application to fail miserably at runtime. All I had to do was to downgrade the amazon-java-aws-core library.

spring-cloud-starter-aws (Edgware.SR2) uses amazon-java-aws-core version 1.11.125 which was causing the AmazonClientException. I had to change the following to my pom file.

	<!-- Exclude aws-java-sdk-core -->
	<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-starter-aws</artifactId>
	<exclusions>
	<exclusion>
	<groupId>com.amazonaws</groupId>
	<artifactId>aws-java-sdk-core</artifactId>
	</exclusion>
	</exclusions>
	</dependency>
	<!-- Include the right version of the dependency -->
	<dependency>
	<groupId>com.amazonaws</groupId>
	<artifactId>aws-java-sdk-core</artifactId>
	<version>1.11.106</version>
	</dependency>

view raw pom.xml hosted with ❤ by GitHub

This will fix the AmazonClientException and enables spring-cloud-starter-aws to auto-configure the AmazonS3 client.

It is as simple as autowiring your ResourcePatternResolver bean and using it to list all the objects inside the bucket.

 

@Autowired
private ResourcePatternResolver resourcePatternResolver;

public List listS3Files(String bucketName) throws IOException {
  final Resource[] resources = resourcePatternResolver
      .getResources("s3://" + bucketName + "/*.xml");

  return Arrays.stream(resources).map(
      resource -> {
        try {
          return resource.getURL().toExternalForm();
        } catch (IOException e) {
          throw new RuntimeException(e);
        }
      }
  ).collect(Collectors.toList());
}

This code example as a project is pushed to my GitHub. You can find the project here.

Spring-batch useful job-repository queries

One confusing thing when starting with spring-batch project is to understand the ability to query the Job repository database and understand what is happening with the job. The awesome datamodel is best modeled to suit well for the spring-batch to keep track of the jobs its managing, but when it comes for someone to query on a job or view the status of jobs, you might have to jump to multiple tables or write up a query joining tables to figure out what is going on. So I decided to use this space to document all the frequent queries that I use when I am viewing the job repository. I will keep updating this space if I find something interesting or useful.

The queries are pretty straight forward and can be easily derived if you properly understand the job repository model described in the web-page here.

PS: My queries may not be the best, but they work for me and I am happy to receive feedback on improving them. So lets get started with the queries now.

Query #1: Query to list all the jobs with job name and parameters of the job.

Often we would want to get the job name along with the parameters with which it was started, when it started etc. So this query below is handy for me when I want to find out what are all the jobs that are in the job repository starting from the most recent one first.

SELECT
je.JOB_EXECUTION_ID, je.JOB_INSTANCE_ID, ji.JOB_NAME, je.START_TIME, je.END_TIME, je.STATUS, bjep.*,je.EXIT_MESSAGE
FROM
BATCH_JOB_EXECUTION je inner join BATCH_JOB_INSTANCE ji inner join BATCH_JOB_EXECUTION_PARAMS bjep
where je.JOB_INSTANCE_ID=ji.JOB_INSTANCE_ID and je.JOB_EXECUTION_ID = bjep.JOB_EXECUTION_ID order by je.JOB_EXECUTION_ID desc;

 

Query#2: List all the steps and its status for a job with a job name

Next comes the step information. Finding out which steps are executing what is the status of each step, to what job is this associated to and to what run is it bound to are questions that are often difficult to comprehend and would require jumps to multiple tables. So this query always comes handy to me when I am having such questions.

select
bse.STEP_EXECUTION_ID,bse.JOB_EXECUTION_ID,ji.JOB_NAME, bse.STEP_NAME, bse.START_TIME, bse.END_TIME,bse.COMMIT_COUNT, bse.READ_COUNT, bse.WRITE_COUNT, bse.STATUS, bse.EXIT_MESSAGE, bse.LAST_UPDATED
from
BATCH_STEP_EXECUTION bse inner join BATCH_JOB_INSTANCE ji inner join BATCH_JOB_EXECUTION je
where ji.JOB_INSTANCE_ID = je.JOB_INSTANCE_ID and bse.JOB_EXECUTION_ID = je.JOB_EXECUTION_ID order by bse.job_execution_id desc;

So far these are the two queries that I use very often to find out more information about the jobs. There are a few more I will update once I feel they serve a very common problem. But if there is something that you might want to add, feel free to leave a comment and I will update the post with those.

 

 

Why I started liking Spring framework?

My journey with Spring framework started not too long ago. It has been just about 6 months I started diving deep into spring, and since then, I have become a huge fan of Spring framework. I got introduced to spring about three years ago. To be honest, many people I talk to, were not inclined to use spring for the following reasons.

• Initial learning curve
• Too much XML code
• Too much magic happening behind the scenes.
• Unwanted complexity of a framework for solving a fairly simple problem.

Now, after I return back to spring after so long, I feel the spring community has done an awesome job in steering the framework to fit the needs of its users.

Lets go thru how spring has shaped itself over the past three years to overcome its so called deficiencies I listed above (Of course according to me).

Initial learning curve:

Spring boot has now all configurations ready to start running a project from the moment we import the project to our local development environment. You do not have to bother about setting up environment, include dependencies and figure out what version you need to use for your project. Everything you initially need, would be bundled together for you in your POM/gradle file. You can just start your development by defining all your components, and everything would wire up automatically for you. I feel spring boot has reduced the initial learning curve for getting started with a spring app.

Too much XML code:

Spring framework now has very good support for Java-config. They have heard the voice of its majority of its users to eradicate XML configs completely in their spring projects. This contributes to a better readability and make the configs and the classes that manage the configuration coupled to the project rather than having a separate file that defines the configs. This in my opinion is the biggest reason for considering to use spring for my project.

Too much magic happening behind the scenes:

Agreed, there is still lot of magic happening behind the scenes, but with a very active community and a rich documentation for spring framework, you would start realizing and appreciating the “MAGIC” that spring does to help you maintain your project.

Unwanted complexity:

It is very true that things like premature-optimizations, inculcating design patterns before the need arises are things that might over complicate a project. But there are a few qualities for a project that are good to have for maintaining the project and are often neglected when starting a project. Things like configs values being decoupled from your application, not instantiating unnecessary objects, dependency injection for better testing are a few things that are good to have right when you start your development. Spring provides many such niceties for you right when you start working on a project.

In my personal opinion, learning spring is not a big deal at all, but I feel one would be able to appreciate its benefits if they know the pain involved in maintaining and writing an enterprise application. More over, having a framework and the terminology used in the framework, would contribute a lot for anyone who is new to the project to quickly grasp the architecture and speed up the development. Spring has definitely done a great job in shaping itself as one of the best enterprise application framework and I would highly encourage anyone trying to start a project to give this awesome framework a though before deciding their project’s technology stack.