Jenkins: Preparing your Shared Library for tests

After explaining how to set up your Shared Library, how to build it and how to run complete pipelines from this Library, it is time to create tests for the behaviour of the Library itself.

This post is part of a Series in which I elaborate on my best practices for running Jenkins at scale which might benefit Agile teams and CI/CD efforts.

As I’ve detailed before, being able to test the behaviour of your Library is instrumental in running Jenkins at scale. You just do not want to have to ‘test’ any changes in your pipelines or underlying code on a live production system. Ever.
The code which is being developed by the team(s) is usually held to very high standards by implementing a strict development process with multiple checks and quality gates: why should the code for your development process be treated differently?

In order for our Jenkins Pipeline code and Shared Library code to be able to be tested, several additions need to be made to the build file.
As per this post, we are building the Library with Gradle. This will be the basis for this post.

Modify build file

The build.gradle from the previous posts is:

apply plugin: 'groovy'
 
// follow the structure as dictated by Jenkins:
sourceSets {
    main {
        groovy {
            srcDirs = ['src','vars']
        }
        resources {
            srcDirs = ['resources']
        }
    }
}
 
repositories {
    jcenter()
    maven {
        url "https://repo.jenkins-ci.org/releases/"
    }
}
 
dependencies {
    compile 'org.codehaus.groovy:groovy-all:2.5.6'
    compile 'org.apache.ivy:ivy:2.4.0'
    compile 'org.jenkins-ci.main:jenkins-core:2.164.1'
    def staplerGAV = 'org.kohsuke.stapler:stapler:1.255'
    compile staplerGAV
    annotationProcessor staplerGAV
    compile 'org.jenkins-ci.plugins.workflow:workflow-step-api:2.19@jar'
    compile 'org.jenkins-ci.plugins:pipeline-utility-steps:2.2.0@jar'
}

Test source code

As the source code for the tests lives in a non-standard location, we need to instruct Gradle where to find it.
For this, you’ll need to add a new sourceSet with the name test:

test {
    groovy {
        srcDirs = ['test']
    }
}

Now, we’ll need to add some more dependencies for the test frameworks, and a little instruction to actually perform tests.

Test frameworks

Wait, is that plural?

Yes, it is! We’ll need at least two, as testing declarative Jenkins pipelines is not a common practice. We’ll use the excellent Jenkins Pipeline Unit by lesfurets in combination with Spock and JUnit 4 & 5 (Vintage).

First up: Spock!

// Declare the dependency for your favourite test framework you want to use in your tests.
// Spock 1.3 for Groovy 2.5
testCompile 'org.spockframework:spock-core:1.3-groovy-2.5'
// allows mocking of classes (in addition to interfaces)
testRuntime 'net.bytebuddy:byte-buddy:1.9.12'
// allows mocking of classes without default constructor (together with CGLIB)
testRuntime 'org.objenesis:objenesis:3.0.1'

Next: Junit 4 & Vintage Engine 5.

// JUnit 4 + Vintage Engine 5
testCompile 'junit:junit:4.12'
testRuntime 'org.junit.vintage:junit-vintage-engine:5.4.1'

And finally: Jenkins Pipeline Unit.

// Jenkins Pipeline Unit testing framework
testCompile 'com.lesfurets:jenkins-pipeline-unit:1.1'

Configure the frameworks

In order for the JUnit tests to be executed, you’ll need to instruct Gradle to run them.

test {
    useJUnitPlatform()
    testLogging {
        events 'passed', 'skipped', 'failed'
    }
}

This will make Gradle use the JUnit test framework, and log the output of test cases when they emit an event like the ones specified.

Final build.gradle

After applying the changes detailed above, your build.gradle would look something like this:

plugin: 'groovy'
 
// follow the structure as dictated by Jenkins:
sourceSets {
    main {
        groovy {
            srcDirs = ['src','vars']
        }
        resources {
            srcDirs = ['resources']
        }
    }
    test {
        groovy {
            srcDirs = ['test']
        }
    }
}
 
repositories {
    jcenter()
    maven {
        url "https://repo.jenkins-ci.org/releases/"
    }
}
 
dependencies {
    compile 'org.codehaus.groovy:groovy-all:2.5.6'
    compile 'org.apache.ivy:ivy:2.4.0'
    compile 'org.jenkins-ci.main:jenkins-core:2.164.1'
    def staplerGAV = 'org.kohsuke.stapler:stapler:1.255'
    compile staplerGAV
    annotationProcessor staplerGAV
    compile 'org.jenkins-ci.plugins.workflow:workflow-step-api:2.19@jar'
    compile 'org.jenkins-ci.plugins:pipeline-utility-steps:2.2.0@jar'
    testCompile 'org.spockframework:spock-core:1.3-groovy-2.5'
    testRuntime 'net.bytebuddy:byte-buddy:1.9.12'
    testRuntime 'org.objenesis:objenesis:3.0.1'
    testCompile 'junit:junit:4.12'
    testRuntime 'org.junit.vintage:junit-vintage-engine:5.4.1'
    testCompile 'com.lesfurets:jenkins-pipeline-unit:1.1'
}

test {
    useJUnitPlatform()
    testLogging {
        events 'passed', 'skipped', 'failed'
    }
}

Add the framework for declarative pipelines

The previously added frameworks are distributed as Maven artefacts and can be obtained via Maven Central or Bintray’s JCenter.
The framework we are going to add now is called pipelineUnit and is only available as Groovy source files, so they need to be copied to your source tree.

We are going to use a fork of the original project, as this is a little more complete than the original. There are some Pull Requests created to merge the added functionality back to the upstream.
The following files need to be copied into the project, at test/groovy/testSupport/:

• PipelineSpockTestBase.groovy
• PipelineTestHelper.groovy
• WhenExitException.groovy

You should now have a structure in your Shared Library like below:

├── build.gradle
├── gradle
│   └── wrapper
│       ├── gradle-wrapper.jar
│       └── gradle-wrapper.properties
├── src
├── test
│   └── groovy
│       └── testSupport
│           ├── PipelineSpockTestBase.groovy
│           ├── PipelineTestHelper.groovy
│           └── WhenExitException.groovy
├── var
│   └── logError.groovy
└── settings.gradle

Now, let’s add a simple test!

With all the setup finally done, it is now time to add tests to your Shared Library.
As dictated by our Gradle build file, tests should live in test, so that is where we’ll create our first file.

Remember the Custom step logError from a previous blog post? Here it is verbatim:

// vars/logError.groovy
def call(String message) {
    // Any valid steps can be called from this code, just like
    // in a Scripted Pipeline
    echo "[ERROR]: ${message}"
}

It just writes your message prepended with a string literal to the console.

Let’s create a test class which:

1. Retrieves the class under test (vars/logError.groovy)
2. Runs the call method
3. Validates that:
   1. Everything succeeds
   2. Only 1 (one) echo is executed to display something in the log
   3. This displayed something includes the text “ERROR”.

// test/logErrorTest.groovy
// Import the required stuff: JUnit & Jenkins Pipeline Unit
import org.junit.*
import static org.junit.Assert.*
import com.lesfurets.jenkins.unit.*

// Extend the BasePipelineTest to use the Jenkins Pipeline Unit framework
class logErrorTest extends BasePipelineTest {
    // The class under test
    def logError

    // Before every testcase is run, do this:
    @Before
    void setUp() {
        super.setUp()
        // Load the script, without executing it.
        logError = loadScript("vars/logError.groovy")
    }

    // This is our testcase!
    @Test
    void 'Log message to console with "ERROR" prepended'() {
        // Execute the 'call' method on our class under test
        logError.call("message")
        // Validate that echo is only called once
        assertEquals(1, helper.methodCallCount('echo'))
        // Validate that the call to echo contains the string "ERROR"
        assertTrue(helper.getCallStack()[1].args[0].toString().contains("ERROR"))
        // print the complete callstack to the console for good measure
        printCallStack()
    }

}

Peeking through this Test-class, you can clearly identify the JUnit style testcase, with a setUp() being run at @Before and a testcase clearly labelled with @Test. This testcase contains two assertions (assertEquals and assertTrue) which validate that the callstack of our class contains the correct information.

When one would run this testcase in IntelliJ IDEA, it would show up like this:

And I would consider that a great success!

Next steps

What I’ve show here so far is only a very simple set up, and it doesn’t even utilize all of the functionality available to us. There is no Spock-style testing, and not a single declarative pipeline has been touched. I’ll show you how to test a complete pipeline in the next blog post!