Layout Testing with Galen, JUnit and Maven

Writing tests not only to verify the behaviour of a web site but also the correctness of its layout especially for responsive websites is not always easy.

Luckily the Galen Framework eases the task of writing layout tests for us, offering a specialized domain-specific-language to write layout-specifications, it integrates well with Selenium Grid, Sauce Labs or BrowserStack, it offers an easy way to deal with different browser sizes and responsive designs and it generates nice, detailed test reports.

In the following tutorial I’m going to implement a JUnit test for the layout of a blog article of mine to demonstrate Galen’s basic features and the integration of it with JUnit and the Maven build tool.

Figure 1. Failed image comparison test report

Dependencies

We just need to add two dependencies to our project’s pom.xml (using Maven):

galen-java-support: The Galen Framework for Java
junit: The JUnit test library (TestNG is supported as well)

<dependencies>
	<dependency>
		<groupId>com.galenframework</groupId>
		<artifactId>galen-java-support</artifactId>
		<version>2.2.5</version>
		<scope>test</scope>
	</dependency>
	<dependency>
		<groupId>junit</groupId>
		<artifactId>junit</artifactId>
		<version>4.12</version>
		<scope>test</scope>
	</dependency>
</dependencies>

Writing a Specification

Galen offers a rich domain-specific-language to write its specifications – it even supports loops and execution of JavaScript functions as well as header and footer import files, if-statements and the ability to write your own custom rules.

The language is documented excellent and very detailed here, please feel free to have a look.

As I’ll be testing one blog article page of mine, "Load Testing Web Applications with Gatling and Maven", this is my – rather simple – specification where..

I’m specifying four elements using CSS or XPath selectors in the @objects section. The name chosen for each element is used to reference it in the further specifications.
I’m creating a group named post_elements in the @groups section, referencing all of the elements that belong to the HTML container for a blog article on my website.
I’m creating a section (may be folded/unfolded in the test results) named Page Header Section
I’m writing a rule that says that the page_heading element must have a width of roundabout 20% of the main_container element and must contain the text “hasCode.com“
I’m creating a second section named Post Section
I’m writing a rule that says that all elements of the group named post_elements (specified in step 2) must be visible.
I’m adding a rule that the element post_heading must contain the text “Maven“_(me remember, the article’s name is “Load Testing Web Applications with Gatling and _Maven“)
I’m adding a rule that says that the element post_image (it’s an image) must match a reference image file stored in the project, named reference-image.png and it must only differ in 5%. To demonstrate the image comparison functions of Galen and the nice reports it produces here, we’re using an image that differs in more than 5% from the original image.
The specification is saved in a file named hascodeArticle.spec in the directory specs (src/test/resources/specs in my Mavenized project..)

@objects
	main_container	xpath      //*[@class='Main']
	page_heading	css	.logo-name a
	post_heading	css	.post h2
	post_image		css #attachment_1292 img

@groups
	post_elements post_heading, post_image

= Page Header Section =
	page_heading:
		width ~ 20 % of main_container/width
		text is "hasCode.com"

= Post Section =
	&post_elements:
		visible

	post_heading:
		text contains "Maven"

	post_image:
		image file reference-image.png, error 5%

Setting up a Test with JUnit and WebDriver

In the following example, I’m implementing a JUnit test using the specification described above and running with different screen sizes.

We simply extend our test’s class from com.galenframework.juni.GalenJUnitTestBase. Our test is now a parameterized test, so that for each object passed by our static provider method, annotated with @Parameterized.Parameters, our tests are run.

For more examples using JUnit’s parameterized API, please feel free to have a look at my article: New features in JUnit 4.11 - Parameterized Tests.

That allows us to pass in different settings for our tests like screen sizes etc. We encapsulate this information in a simple helping POJO named DeviceSetup and provide three instances of it as test parameters.

To keep it simple, we’re using the FirefoxDriver implementation of the WebDriver API here.

We’re loading the blog article page using the provided load() method and we’re testing with our specification using the provided checkLayout() method.

The WebDriver API may be used as we know it to navigate between pages or use page objects – if you’re interested in this topic, please feel free to have a look at another blog article of mine: Selenium WebDriver, Selenium Server and PageObjects by Example.

package it;

import java.util.Arrays;
import java.util.List;

import org.junit.Test;
import org.junit.runners.Parameterized;
import org.openqa.selenium.Dimension;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;

import com.galenframework.junit.GalenJUnitTestBase;

public class BlogArticlePageTest extends GalenJUnitTestBase {

	@Override
	public WebDriver createDriver() {
		return new FirefoxDriver();
	}

	private DeviceSetup device;

	public BlogArticlePageTest(final DeviceSetup deviceSetup) {
		super();
		this.device = deviceSetup;
	}

	public static class DeviceSetup {
		private final Dimension screenSize;
		private final List<String> tags;

		public DeviceSetup(Dimension screenSize, String... tags) {
			this.screenSize = screenSize;
			this.tags = Arrays.asList(tags);
		}

		public Dimension getScreenSize() {
			return screenSize;
		}

		protected List<String> getTags() {
			return tags;
		}
	}

	@Parameterized.Parameters
	public static Iterable<Object[]> devices() {
		return Arrays.asList(new Object[][] { { new DeviceSetup(new Dimension(1024, 800), "normal", "desktop") },
				{ new DeviceSetup(new Dimension(280, 800), "small-phone", "phone", "mobile") },
				{ new DeviceSetup(new Dimension(320, 800), "normal-phone", "phone", "mobile") }

		});
	}

	@Test
	public void shouldRenderBlogArticleCorrect() throws Exception {
		load("https://www.hascode.com/2016/05/load-testing-web-applications-with-gatling-and-maven/",
				device.getScreenSize().getWidth(), device.getScreenSize().getHeight());
		checkLayout("/specs/hascodeArticle.spec", device.getTags());
	}
}

Testing a Website Layout

We may now run our tests using Maven and the command line:

$ mvn test
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running it.BlogArticlePageTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 73.516 sec
[main] INFO com.galenframework.junit.JUnitStepListener - Generating Galen Html reports

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

Or with some failures..

$ mvn test
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running it.BlogArticlePageTest
Tests run: 3, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 286.841 sec <<< FAILURE!
[main] INFO com.galenframework.junit.JUnitStepListener - Generating Galen Html reports

Results :

Tests in error:
  shouldRenderBlogArticleCorrect[0](it.BlogArticlePageTest): /specs/hascodeArticle.spec, tags: [normal, desktop]
  shouldRenderBlogArticleCorrect[1](it.BlogArticlePageTest): /specs/hascodeArticle.spec, tags: [small-phone, phone, mobile]
  shouldRenderBlogArticleCorrect[2](it.BlogArticlePageTest): /specs/hascodeArticle.spec, tags: [normal-phone, phone, mobile]

Tests run: 3, Failures: 0, Errors: 3, Skipped: 0

Test Reports

When all tests are run, a surefire-report is generated as well as a more specific Galen report – it is written per default to the target/galen directory.

Test Overview

The main page of the generated reports displays a short overview of all test permutations run and their result.

The following screenshot displays the result of our parameterized tests above.

Figure 2. Galen Test Results Overview

Detailed Specification Report

In the following detailed report, each section and rule of our specification is displayed and we may jump to other reports from here like the heat map or image comparison result views.

Figure 3. Galen Test Run Detail Report

Heat Map

The heat map shows, which elements of our web page are covered by specifications. In a production scenario, we should hopefully see a higher coverage in our heat map ;)

Figure 4. Galen Heat Map View

Image Comparison Result

As our images differ slightly in size, they produce an error. The image comparison displays differences between actual image and expectation.

Figure 5. Failed image comparison test report

Tutorial Sources

Please feel free to download the tutorial sources from my GitHub repository, fork it there or clone it using Git:

git clone https://github.com/hascode/galen-testing-tutorial.git

Layout Testing with Galen, JUnit and Maven

Dependencies

Writing a Specification

Setting up a Test with JUnit and WebDriver

Testing a Website Layout

Test Reports

Test Overview

Detailed Specification Report

Heat Map

Image Comparison Result

Tutorial Sources

Resources

Other Testing Articles of mine

Dependencies#

Writing a Specification#

Setting up a Test with JUnit and WebDriver#

Testing a Website Layout#

Test Reports#

Test Overview#

Detailed Specification Report#

Heat Map#

Image Comparison Result#

Tutorial Sources#

Resources#

Other Testing Articles of mine#

Dependencies

Writing a Specification

Setting up a Test with JUnit and WebDriver

Testing a Website Layout

Test Reports

Test Overview

Detailed Specification Report

Heat Map

Image Comparison Result

Tutorial Sources

Resources

Other Testing Articles of mine