Implementing, Testing and Running Procedures for Neo4j

A lot of features are already included in the Neo4j graph database system but sometimes we want to extends its capabilities and implement functions and procedures by ourselves that we may reuse.

In the following tutorial I will demonstrate how to implement a procedure for Neo4j, how to write and run tests using JUnit and an embedded graph database and last but not least how to setup Neo4j with Docker and our stored procedure installed in no time.

Figure 1. Calling a stored procedure in the Neo4j browser.

Implementing a Stored Procedure

For the purpose of this tutorial we’ll be implementing a stored procedure to fetch quality metrics from a graph. We’ll be using the graph model generated by jqAssistant here to calculate the abstractness of a given package.

For more detailed information about scanning and analyzing Java projects with jqAssistant and Neo4j please feel free to read my tutorial: “Software Architecture Exploration and Validation with jqAssistant, Neo4j and Cypher“.

The abbreviated graph model could be reduces to the following representation:

(:Package)-[:CONTAINS]→(:Type:Class)

(:Package)-[:CONTAINS]→(:Type:Interface)

Where Types have an attribute fqn that contains the full-qualified-name .. e.g. com.hascode.SomeClass.

Project Setup

Just one dependency needs to be declared when using Maven by adding the following line to our pom.xml:

<dependency>
    <groupId>org.neo4j</groupId>
    <artifactId>neo4j</artifactId>
    <version>3.3.2</version>
    <scope>provided</scope>
</dependency>

Cypher Query

This is a quick and dirty example of our cypher query to calculate the abstractness of a package.

Parameters are enclosed in curly brackets.

MATCH (t:Type)
WHERE t.fqn STARTS WITH {packageFqn}
AND (t.abstract IS NULL
OR NOT t:Interface)
WITH COUNT(t) AS nonabstract
MATCH (t:Type)
WHERE t.fqn STARTS WITH {packageFqn}
AND (t.abstract = TRUE
OR t:Interface)
WITH nonabstract, COUNT(t) AS abstract
RETURN toFloat(abstract) / toFloat(nonabstract) AS abstractness

Procedure

This is our stored procedure.

Necessary accessors like the GraphDatabaseService or the Log may be injected using the @Context annotation.

Our procedure is marked using the @Procedure annotation, the name is the name that we’ll be using to call the procedure later and procedures should return a Stream of values.

In addition we may specify the access mode of our procedure .. e.g. if it writes to the graph database or - as in our case - it just reads information from the database.

For more detailed information about writing procedures, the possible return types and access modes, I’d highly recommend consulting the Neo4j manual here.

package com.hascode.neo4j;

import java.util.HashMap;
import java.util.Map;
import java.util.Objects;
import java.util.stream.Stream;
import org.neo4j.graphdb.GraphDatabaseService;
import org.neo4j.graphdb.ResourceIterator;
import org.neo4j.graphdb.Result;
import org.neo4j.graphdb.Transaction;
import org.neo4j.logging.Log;
import org.neo4j.procedure.Context;
import org.neo4j.procedure.Mode;
import org.neo4j.procedure.Name;
import org.neo4j.procedure.Procedure;

public class ArchitectureMetrics {

  private static final String ABSTRACTNESS_QUERY = "MATCH (t:Type)\n"
      + "WHERE t.fqn STARTS WITH {packageFqn}\n"
      + "AND (t.abstract IS NULL\n"
      + "OR NOT t:Interface)\n"
      + "WITH COUNT(t) AS nonabstract\n"
      + "MATCH (t:Type)\n"
      + "WHERE t.fqn STARTS WITH {packageFqn}\n"
      + "AND (t.abstract = TRUE\n"
      + "OR t:Interface)\n"
      + "WITH nonabstract, COUNT(t) AS abstract\n"
      + "RETURN toFloat(abstract) / toFloat(nonabstract) AS abstractness";

  public static class Abstractness {

    public final double abstractness;

    public Abstractness(double abstractness) {
      this.abstractness = abstractness;
    }
  }

  @Context
  public GraphDatabaseService db;

  @Context
  public Log log;

  @Procedure(name = "hascode.abstractnessForPackage", mode = Mode.READ)
  public Stream<Abstractness> abstractnessForPackage(@Name("packageFqn") String packageFqn) {
    Objects.requireNonNull(packageFqn, "packageFqn must not be null");

    log.info("call hascode.abstractnessForPackage for package: `%s`", packageFqn);

    Map<String, Object> params = new HashMap<>();
    params.put("packageFqn", packageFqn);

    Double abstractness = -1D; // cheesy fallback
    try (Transaction tx = db.beginTx(); Result result = db.execute(ABSTRACTNESS_QUERY, params)) {
      ResourceIterator<Object> resourceIterator = result.columnAs("abstractness");
      if (resourceIterator.hasNext()) {
        abstractness = (Double) resourceIterator.next();
      }
      tx.success();
    }
    return Stream.of(new Abstractness(abstractness));
  }
}

Testing

Having written a procedure is nice but worthless without having tests to prove its correctness.

Therefore we’ll be writing a test for our procedure using the well known JUnit testing library and neo4j-harness.

The latter provides us with convenience methods and JUnit test rules to control an embedded Neo4j database for testing.

Project Setup

To write our test we need additional dependencies in our pom.xml:

<dependency>
    <groupId>org.neo4j.test</groupId>
    <artifactId>neo4j-harness</artifactId>
    <version>3.3.2</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.neo4j.driver</groupId>
    <artifactId>neo4j-java-driver</artifactId>
    <version>1.5.0</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.12</version>
    <scope>test</scope>
</dependency>

Setup Cypher Query

To test our stored procedure, we need to setup some test data first.

In our case, we’re just adding three nodes .. 1 abstract class, 1 class and one interface.

CREATE (c1:Type:Class{name:'Foo', fqn:'com.hascode.Foo', abstract:true}),
(c2:Type:Class{name:'Bar', fqn:'com.hascode.Bar'}),
(c3:type:Interface{name:'IBaz', fqn:'com.hascode.Baz'})
RETURN c1,c2,c3

This is what our query looks like in the Neo4j browser:

Figure 2. Creating test data

JUnit Test

Now to our JUnit test .. we’re using the Neo4jRule test rule to initialize the system and add our procedure.

Then we’re creating the nodes using the query described above.

Finally we’re calling our stored procedure and we’re verifying the result.

package it;

import static org.hamcrest.CoreMatchers.equalTo;
import static org.hamcrest.MatcherAssert.assertThat;

import com.hascode.neo4j.ArchitectureMetrics;
import org.junit.Rule;
import org.junit.Test;
import org.neo4j.driver.v1.Config;
import org.neo4j.driver.v1.Driver;
import org.neo4j.driver.v1.GraphDatabase;
import org.neo4j.driver.v1.Session;
import org.neo4j.driver.v1.StatementResult;
import org.neo4j.harness.junit.Neo4jRule;

public class ArchitectureMetricsTest {

  @Rule
  public Neo4jRule neo4j = new Neo4jRule().withProcedure(ArchitectureMetrics.class);

  @Test
  public void mustCalculatePackageAbstractness() throws Exception {
    try (Driver driver = GraphDatabase
        .driver(neo4j.boltURI(), Config.build().withoutEncryption().toConfig())) {
      Session session = driver.session();
      session.run(
          "CREATE (c1:Type:Class{name:'Foo', fqn:'com.hascode.Foo', abstract:true}),"
              + "(c2:Type:Class{name:'Bar', fqn:'com.hascode.Bar'}),"
              + "(c3:type:Interface{name:'IBaz', fqn:'com.hascode.Baz'})"
              + "RETURN c1,c2,c3");

      StatementResult result = session
          .run("CALL hascode.abstractnessForPackage('com.hascode')");

      final Double abstractness = result.single().get("abstractness").asDouble();
      assertThat("abstractness of package com.hascode should be 0.5", abstractness, equalTo(0.5));
    }
  }

}

Running the Test

We may now run our tests in the IDE of choice or in the command-line like this:

$ mvn test
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running it.ArchitectureMetricsTest
Feb 26, 2018 9:47:55 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25 AM'
Feb 26, 2018 9:47:55 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25 AM'
Feb 26, 2018 9:47:55 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25 AM'
Feb 26, 2018 9:47:56 PM org.neo4j.driver.internal.logging.JULogger info
INFO: Driver instance org.neo4j.driver.internal.InternalDriver@58aa10f4 created
Feb 26, 2018 9:47:58 PM org.neo4j.driver.internal.logging.JULogger info
INFO: Closing driver instance org.neo4j.driver.internal.InternalDriver@58aa10f4
Feb 26, 2018 9:47:58 PM org.neo4j.driver.internal.logging.JULogger info
INFO: Closing connection pool towards 127.0.0.1:35065
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.689 sec - in it.ArchitectureMetricsTest

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

Running with Docker

Another alternative is to test our stored procedure using a Dockerized Neo4j instance.

Luckily for us there is an official Docker image on DockerHub that we may use.

The following command starts our Docker image, includes a local directory target/plugintmp as the plugin directory and exports the necessary ports.

docker run -td --rm -v $PWD/target/plugintmp:/plugins -p 7474:7474 -p 7687:7687 neo4j:3.3.2

From our project directory, we may build and package our procedure, copy it to this directory and start the Neo4j Docker instance with our procedure loaded.

mvn clean package -Dmaven.test.skip=true && \
mkdir target/plugintmp && \
cp target/neo4j-stored-procedures-1.0.0.jar target/plugintmp && \
docker run -td --rm -v $PWD/target/plugintmp:/plugins -p 7474:7474 -p 7687:7687 neo4j:3.3.2

Now that the instance has started, we may use our procedure.

The following command enlists all existing procedures – we should see a procedure named “hascode.abstractnessForPackage“.

CALL dbms.procedures()

This is what the procedures list looks like in the Neo4j browser:

Figure 3. Display existing stored procedures

Tutorial Sources

Please feel free to download the tutorial sources from my GitHub repository, fork it there or clone it using Git:

git clone https://github.com/hascode/neo4j-stored-procedures.git

Resources

Appendix A: APOC

APOC stands for “Awesome Procedures On Cypher” and the name is program.

There are multiple extremely helpful procedures like loading and processing JSON, encryption functions, date/time-functions, math-functions and more .. a good overview can be found here.

Implementing a Stored Procedure#

Project Setup#

Cypher Query#

Procedure#

Testing#

Project Setup#

Setup Cypher Query#

JUnit Test#

Running the Test#

Running with Docker#

Tutorial Sources#

Resources#

Other Neo4j Tutorials#

Appendix A: APOC#

Implementing a Stored Procedure

Project Setup

Cypher Query

Procedure

Testing

Project Setup

Setup Cypher Query

JUnit Test

Running the Test

Running with Docker

Tutorial Sources

Resources

Other Neo4j Tutorials

Appendix A: APOC