Creating In-Memory File Systems with Google’s Jimfs

Sometimes when writing an application we might consider using an in-memory file system to speed up data access or to create some kind of cache.

There are different libraries to help us here but one looks especially promising for me because it supports almost every functionality of the Java NIO File APIs added in Java 7 – from creating, reading, deleting files and directory to handling symbolic and hard links or watching directory changes with a WatchService.

In the following short tutorial, I’d like to demonstrate how to setup an in-memory file system within a few minutes and how to access directories and files stored in this file system.

Figure 1. Creating an in-memory file system with Jimfs

Dependencies

Only one dependency needs to be added to our project: com.google.jimfs:jimfs:1.0

Using Gradle as build tool, this is my build.gradle:

apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'application'

sourceCompatibility = 1.8
version = '1.0.0'
jar {
    manifest {
        attributes 'Implementation-Title': 'hasCode.com Jimfs Tutorial', 'Implementation-Version': version
    }
}

repositories {
    mavenCentral()
}

dependencies {
    compile 'com.google.jimfs:jimfs:1.0'
}

mainClassName = 'com.hascode.tutorial.Example'

In-Memory File System and the NIO API

In the following examples, we’re creating an in-memory file system and we’re accessing it using the NIO API added with Java 7.

Example 1: Creating and Reading Directories and Files

First we’re writing two new files to the file system, we’re copying one file from the classpath to the file system and finally we’re reading all files from the in-memory file system’s data directory and print their name and byte-length.

package com.hascode.tutorial;

import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.nio.file.FileSystem;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;

import com.google.common.collect.ImmutableList;
import com.google.common.jimfs.Configuration;
import com.google.common.jimfs.Jimfs;

public class Example {

	public static void main(final String[] args) throws IOException {
		System.out.println("Example 1: Creating, copying, reading files and directories");
		FileSystem fs = Jimfs.newFileSystem(Configuration.unix());
		Path data = fs.getPath("/data");
		Files.createDirectory(data);

		Path hello = data.resolve("test.txt"); // /data/test.txt
		Files.write(hello, ImmutableList.of("hello world"), StandardCharsets.UTF_8);

		Path csv = data.resolve("data.csv"); // /data/data.csv
		Files.write(csv, ImmutableList.of("test1,test2\ntest3,test4"), StandardCharsets.UTF_8);

		InputStream istream = Example.class.getResourceAsStream("/book.xml");
		Path xml = data.resolve("book.xml"); // /data/book.xml
		Files.copy(istream, xml, StandardCopyOption.REPLACE_EXISTING);

		Files.list(data).forEach(file -> {
			try {
				System.out.println(String.format("%s (%db)", file, Files.readAllBytes(file).length));
			} catch (Exception e) {
				e.printStackTrace();
			}
		});
	}
}

When we’re running the code above using our IDE of choice or using Gradle we should get a similar output:

$ gradle run
[..]
Example 1: Creating, copying, reading files and directories
/data/book.xml (146b)
/data/data.csv (24b)
/data/test.txt (12b)

BUILD SUCCESSFUL

Total time: 3.898 secs

Example 2: Creating and Reading Symbolic Links

In the following example, we’re creating a symlink to the file test.txt:

package com.hascode.tutorial;

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.FileSystem;
import java.nio.file.Files;
import java.nio.file.Path;

import com.google.common.collect.ImmutableList;
import com.google.common.jimfs.Configuration;
import com.google.common.jimfs.Jimfs;

public class Example {

	public static void main(final String[] args) throws IOException {
		System.out.println("Example 2: Handling Symbolic Links");
		FileSystem fs = Jimfs.newFileSystem(Configuration.unix());
		Path data = fs.getPath("/data");
		Files.createDirectory(data);

		Path hello = data.resolve("test.txt"); // /data/test.txt
		Files.write(hello, ImmutableList.of("hello world"), StandardCharsets.UTF_8);

		Path linkToHello = data.resolve("test.txt.link");
		Files.createSymbolicLink(linkToHello, hello);

		Files.list(data).forEach(file -> {
			try {
				System.out.println(String.format("%s (%db)", file, Files.readAllBytes(file).length));
			} catch (Exception e) {
				e.printStackTrace();
			}
		});
	}

}

When we’re running the code above using our IDE of choice or using Gradle we should get a similar output:

$ gradle run
[..]
Example 2: Handling Symbolic Links
/data/test.txt (12b)
/data/test.txt.link (12b)

BUILD SUCCESSFUL

Total time: 3.594 secs

Example 3: Watching for Changes using a WatchService

In the third example, we’re going to configure a watch-service to monitor changes in our virtual in-memory file system.

package com.hascode.tutorial;

import static java.nio.file.StandardWatchEventKinds.ENTRY_CREATE;
import static java.nio.file.StandardWatchEventKinds.ENTRY_MODIFY;

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.FileSystem;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.WatchEvent;
import java.nio.file.WatchKey;
import java.nio.file.WatchService;

import com.google.common.collect.ImmutableList;
import com.google.common.jimfs.Configuration;
import com.google.common.jimfs.Jimfs;

public class Example {

	public static void main(final String[] args) throws IOException, InterruptedException {
		System.out.println("Example 3: Watching changes using a WatchService");
		FileSystem fs = Jimfs.newFileSystem(Configuration.unix());
		Path data = fs.getPath("/data");
		Files.createDirectory(data);

		WatchService watcher = data.getFileSystem().newWatchService();
		Thread watcherThread = new Thread(() -> {
			WatchKey key;
			try {
				key = watcher.take();
				while (key != null) {
					for (WatchEvent<?> event : key.pollEvents()) {
						System.out.printf("event of type: %s received for file: %s\n", event.kind(), event.context());
					}
					key.reset();
					key = watcher.take();
				}
			} catch (Exception e) {
				e.printStackTrace();
			}
		}, "CustomWatcher");
		watcherThread.start();
		data.register(watcher, ENTRY_CREATE, ENTRY_MODIFY);

		Path hello = data.resolve("test.txt"); // /data/test.txt
		Files.write(hello, ImmutableList.of("hello world"), StandardCharsets.UTF_8);
	}
}

Running the code above should produce a similar output:

$ gradle run
[..]
Example 3: Watching changes using a WatchService
event of type: ENTRY_CREATE received for file: test.txt

Tutorial Sources

Please feel free to download the tutorial sources from my GitHub repository, fork it there or clone it using Git:

git clone https://github.com/hascode/jimfs-tutorial.git

Resources

Article Updates

2015-03-20: Example using a WatchService added.

Dependencies#

In-Memory File System and the NIO API#

Example 1: Creating and Reading Directories and Files#

Example 2: Creating and Reading Symbolic Links#

Example 3: Watching for Changes using a WatchService#

Tutorial Sources#

Resources#

Article Updates#