When it comes to web-services (especially RESTful web-services), configuration files or other data-descriptors, the JavaScript Object Notation, short: JSON is often used as format of choice.

As the task arises to transform these JSON structures, I have seen a variety of different approaches from converting JSON into XML using JAX-B, applying XSLT,  transforming into JSON again on the one hand to loading JSON structures into Hash-maps of Hash-maps (..) and manually removing single elements from these collections on the other hand.

JOLT is a library to make this task easier for us: It allows us to note down different types of transformations in a specification file in JSON syntax and to apply this specification to given JSON structures with ease.

In the following short tutorial I’d like to demonstrate how to use this library by transforming JSON structures programmatically with a few lines of code, by filtering a RESTful web-service using JOLT and a servlet filter and finally by adding a short example using the Apache Camel framework.

rest service servlet filtered 1024x799
Figure 1. Filtered RESTful Webservice output

JOLT Schema Syntax

Jolt allows the following transformations that may be combined to generate the result we need. Please consult the linked JavaDocs for more details about the concrete syntax. In the following examples, we’ll be using only one simple remove command to remove elements from a given JSON structure.

In addition, custom Java implementations may be used, they simply need to implement Transform or ContextualTransform.

Programmatic Transformation

In our first, simple example, we’re loading our transformer specification from a JSON file in the classpath and apply the transformation to a JSON structure read from another file.

Finally we’re printing the result to the screen.

Dependencies

We just need to add two dependencies to our project’s pom.xml (using Maven): jolt-core and jolt-utils:

<dependency>
    <groupId>com.bazaarvoice.jolt</groupId>
    <artifactId>jolt-core</artifactId>
    <version>0.1.0</version>
</dependency>
<dependency>
    <groupId>com.bazaarvoice.jolt</groupId>
    <artifactId>json-utils</artifactId>
    <version>0.1.0</version>
</dependency>

JSON Structure

This is the JSON structure we’ll be using for all following demonstrations. In all examples, we’re reducing the structure by removing the page, date, startTime and endTime elements.

[
   {
      "uuid":"cac40601-ffc9-4fd0-c5a1-772ac65f0587",
      "pageId":123456,
      "location":"Berlin",
      "maxParticipants":10,
      "page":{
         "indexable":true,
         "rootLevel":false,
         "homePage":false,
         "latestVersion":true,
         "current":true,
         "deleted":false,
         "draft":false,
         "unpublished":false,
         "versionCommentAvailable":false,
         "persistent":true,
         "new":false
      },
      "date":"15.02.17",
      "startTime":"09:00",
      "endTime":"16:30",
      "start":1487145600000,
      "end":1487172600000,
      "hasStartTime":true,
      "hasEndTime":true,
      "eventPageUrl":"/some/url",
      "categories":[
         {
            "label":"Test",
            "id":123,
            "belongsToEvents":true
         }
      ],
      "participants":[

      ],
      "isCurrentUserParticipating":false,
      "fullTitle":"Test Page"
   }
]

JOLT Schema Specification

This is our transformation schema. In this schema we may specify one or multiple operations using JSON notation.

In this schema we’re just applying a remove operation to every sub-node in a given JSON structure:

[
    {
        "operation": "remove",
        "spec":    {
          "*": {
           "page":"",
           "date":"",
           "startTime":"",
           "endTime":""
         }
        }
    }
]

Application

Finally is this our application that load the transformation specs and the JSON input from the classpath, applies the transformation and prints the result to STDOUT:

package com.hascode.tutorial;

import java.util.List;

import com.bazaarvoice.jolt.Chainr;
import com.bazaarvoice.jolt.JsonUtils;

public class CustomJsonTransformer {
    public static void main(String[] args) throws Exception {
        List<Object> specs = JsonUtils.classpathToList("/spec.json");
        Chainr chainr = Chainr.fromSpec(specs);

        Object inputJSON = JsonUtils.classpathToObject("/input.json");
        Object transformedOutput = chainr.transform(inputJSON);
        System.out.println(JsonUtils.toPrettyJsonString(transformedOutput));
    }
}

Running the Example

We may now run our application using our IDE or directly in the command-line using Maven:

$ mvn clean compile exec:java -Dexec.mainClass=com.hascode.tutorial.CustomJsonTransformer
[..]
[ {
  "uuid" : "cac40601-ffc9-4fd0-c5a1-772ac65f0587",
  "pageId" : 123456,
  "location" : "Berlin",
  "maxParticipants" : 10,
  "start" : 1487145600000,
  "end" : 1487172600000,
  "hasStartTime" : true,
  "hasEndTime" : true,
  "eventPageUrl" : "/some/url",
  "categories" : [ {
    "label" : "Test",
    "id" : 123,
    "belongsToEvents" : true
  } ],
  "participants" : [ ],
  "isCurrentUserParticipating" : false,
  "fullTitle" : "Test Page"
} ]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

Filtering a RESTful Webservice using Servlet Filters

In the next example, we’re setting up a JAX-RS compliant RESTful web-service using an embedded Jetty server.

This endpoint sends the JSON structure in a GET request and a JOLT empowered servlet filter transforms this output by applying a transformation.

Dependencies

We’re adding a bunch of dependencies for Jersey (JAX-RS), Servlet-Container, Servlet-API and so on..

<properties>
    <jetty.version>9.4.1.v20170120</jetty.version>
    <jersey.version>2.25.1</jersey.version>
</properties>

<dependency>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-server</artifactId>
    <version>${jetty.version}</version>
</dependency>
<dependency>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-servlet</artifactId>
    <version>${jetty.version}</version>
</dependency>
<dependency>
    <groupId>org.glassfish.jersey.core</groupId>
    <artifactId>jersey-server</artifactId>
    <version>${jersey.version}</version>
</dependency>
<dependency>
    <groupId>org.glassfish.jersey.containers</groupId>
    <artifactId>jersey-container-servlet-core</artifactId>
    <version>${jersey.version}</version>
</dependency>
<dependency>
    <groupId>org.glassfish.jersey.containers</groupId>
    <artifactId>jersey-container-jetty-http</artifactId>
    <version>${jersey.version}</version>
    <exclusions>
        <exclusion>
            <groupId>org.eclipse.jetty</groupId>
            <artifactId>jetty-util</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.glassfish.jersey.media</groupId>
    <artifactId>jersey-media-moxy</artifactId>
    <version>${jersey.version}</version>
</dependency>

REST Resource

This is our REST resource reading in our JSON file from the example above and sending it in a GET-request:

package com.hascode.tutorial;

import java.io.IOException;
import java.net.URISyntaxException;
import java.nio.file.Files;
import java.nio.file.Paths;

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;

@Path("/")
public class SampleRestResource {

    @GET
    @Path("/data")
    @Produces(MediaType.APPLICATION_JSON)
    public Response getData() {
        try {
            String json = new String(Files.readAllBytes(Paths.get(getClass().getClassLoader().getResource("input.json").toURI())));
            return Response.ok(json).build();
        } catch (IOException | URISyntaxException e) {
        }
        return Response.serverError().build();
    }
}

Jetty JAX-RS Server

This is our application class where we’re starting an embedded Jetty, registering the REST resource as well as the servlet filter and starting a server instance on port 9000:

package com.hascode.tutorial;

import java.util.EnumSet;

import javax.servlet.DispatcherType;

import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.servlet.ServletContextHandler;
import org.eclipse.jetty.servlet.ServletHolder;

public class RestServer {

    public static void main(String[] args) throws Exception {
        ServletContextHandler context = new ServletContextHandler(ServletContextHandler.SESSIONS);
        context.setContextPath("/");

        Server jettyServer = new Server(9000);
        jettyServer.setHandler(context);

        ServletHolder jerseyServlet = context.addServlet(org.glassfish.jersey.servlet.ServletContainer.class, "/*");
        jerseyServlet.setInitParameter("jersey.config.server.provider.classnames",
                SampleRestResource.class.getCanonicalName());
        jerseyServlet.setInitOrder(0);

        context.addFilter(JsonTransformerFilter.class, "/*",
                EnumSet.of(DispatcherType.REQUEST, DispatcherType.FORWARD));

        try {
            jettyServer.start();
            jettyServer.join();
        } finally {
            jettyServer.destroy();
        }
    }

}

Servlet Filter

Our servlet filter reads the specification from the classpath, passes the filter chain with a decorated servlet request wrapper and finally writes the result of the transformation so that we’re able to receive a modified JSON structure in the REST service’s response:

package com.hascode.tutorial;

import java.io.IOException;
import java.io.PrintWriter;
import java.util.List;

import javax.servlet.Filter;
import javax.servlet.FilterChain;
import javax.servlet.FilterConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.http.HttpServletResponse;

import com.bazaarvoice.jolt.Chainr;
import com.bazaarvoice.jolt.JsonUtils;

public class JsonTransformerFilter implements Filter {
    @Override
    public void init(FilterConfig filterConfig) throws ServletException {
        System.out.println("json transformer servlet filter initialized");
    }

    @Override
    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
            throws IOException, ServletException {
        List<Object> specs = JsonUtils.classpathToList("/spec.json");
        System.out.println("specs loaded:");
        specs.forEach(System.out::println);
        Chainr chainr = Chainr.fromSpec(specs);

        CustomResponseWrapper res = new CustomResponseWrapper((HttpServletResponse) response);
        chain.doFilter(request, res);

        String inputJSON = res.getContent();
        Object transformedOutput = chainr.transform(JsonUtils.jsonToObject(inputJSON));
        System.out.printf("transformed json: %s\n", transformedOutput);

        PrintWriter writer = response.getWriter();
        writer.write(transformedOutput.toString());
        writer.flush();
    }

    @Override
    public void destroy() {
    }

}

Servlet Response Wrapper

Just a simple decorator to wrap the servlet response, if interested, have a look at my quick implementation here.

Running the Service

Now we’re ready to start our RESTful webservice in our IDE or command line using Maven like this:

$ mvn clean compile exec:java -Dexec.mainClass=com.hascode.tutorial.RestServer
[INFO] ------------------------------------------------------------------------
[INFO] Building json-transform-jolt-tutorial 1.0.0
[INFO] ------------------------------------------------------------------------
[..]
json transformer servlet filter initialized
specs loaded:
{operation=remove, spec={*={page=, date=, startTime=, endTime=}}}
transformed json: [{uuid=cac40601-ffc9-4fd0-c5a1-772ac65f0587, pageId=123456, location=Berlin, maxParticipants=10, start=1487145600000, end=1487172600000, hasStartTime=true, hasEndTime=true, eventPageUrl=/some/url, categories=[{label=Test, id=123, belongsToEvents=true}], participants=[], isCurrentUserParticipating=false, fullTitle=Test Page}]
specs loaded:
{operation=remove, spec={*={page=, date=, startTime=, endTime=}}}

Now we should be able to access our REST service pointing our browser to http://localhost:9000/data

rest service servlet filtered 1024x799
Figure 2. Filtered RESTful Webservice output

JOLT and Apache Camel

Apache Camel is a nice framework to apply different enterprise integration patterns and to allow our applications to use and connect endpoints in every protocol we can imagine – and of course there is also a component for JOLT transformations support.

I have written articles about Apache Camel in the past, please feel free to read further if interested:

Dependencies

We simply need to add two dependencies to our project: camel-core and camel-jolt:

<properties>
    <camel.version>2.18.1</camel.version>
</properties>

<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-core</artifactId>
    <version>${camel.version}</version>
</dependency>
<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-jolt</artifactId>
    <version>${camel.version}</version>
</dependency>

Sample Application

This is our final application..

  • starting a camel route

  • reading from the input directory named scanned

  • searching for files named input.json

  • passing them to the JOLT transformer

  • applying the transformation from our specification file named spec.json

  • writing the transformed file to the output directory named transformed.

package com.hascode.tutorial;

import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.main.Main;

public class CamelExample {

    public static void main(String[] args) throws Exception {
        Main main = new Main();
        main.addRouteBuilder(new RouteBuilder() {
            @Override
            public void configure() throws Exception {
                from("file://scanned?fileName=input.json")
                        .to("jolt:/spec.json?inputType=JsonString&outputType=JsonString").to("file://transformed");
            }
        });

        main.run();
    }

}

Running the Example

Again we may now run our application in our IDE/from command line:

$ mvn clean compile exec:java -Dexec.mainClass=com.hascode.tutorial.CamelExample

When we now copy the sample input.json into the directory named scanned we should afterwards see a transformed JSON file in the output directory named transformed.

Resources

Tutorial Sources

Please feel free to download the tutorial sources from my GitHub repository, fork it there or clone it using Git:

git clone https://github.com/hascode/json-transform-jolt-tutorial.git

Troubleshooting

  • “java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NoClassDefFoundError: org/eclipse/jetty/util/Decorator at com.hascode.tutorial.RestServer.main(RestServer.java:14) … 6 more Caused by: java.lang.ClassNotFoundException: org.eclipse.jetty.util.Decorator at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) … 7 more” → The dependency jersey-container-jetty-http uses an older version of jetty-util, adding an exclusion to this dependency solves the problem:

    <dependency>
    <groupId>org.glassfish.jersey.containers</groupId>
    <artifactId>jersey-container-jetty-http</artifactId>
    <version>${jersey.version}</version>
    <exclusions>
        <exclusion>
            <groupId>org.eclipse.jetty</groupId>
            <artifactId>jetty-util</artifactId>
        </exclusion>
    </exclusions>
</dependency>