In the previous posts, I’ve shown how to set up your Jenkins Shared Library, create Custom Pipeline Steps in it, set up the test frameworks, run complete pipelines from your Library and write tests for your Custom Pipeline Steps. Now the time has come to test the full declarative pipelines!
This post is part of a Series in which I elaborate on my best practices for running Jenkins at scale which might benefit Agile teams and CI/CD efforts.
Remember why we are going through all this trouble? We want to have as little issues as possible during the development of functionality in pipelines and steps, and want to achieve this through:
- Documenting the behaviour of pipelines/steps in tests
- Validating the current implementation of pipelines/steps
- Removing the need for a ‘live’ environment through mocking
- Enabling the possibility for debugging the code in an IDE
So, without further ado, I’ll show how to create and run a test for a full declarative pipeline.
Full declarative pipeline
Let’s assume you have a declarative-style pipeline in your Shared Library like this one:
// vars/defaultPipeline.groovy
// This pipeline requires no parameters as input.
def call(Map pipelineParams) {
pipeline {
agent none
stages {
stage('Build and Unit test') {
agent { label 'maven' }
steps {
script {
module_Maven('clean verify')
}
}
post {
always {
junit testResults: '**/target/surefire-reports/*.xml', allowEmptyResults: false
}
}
}
stage('Publish to Nexus') {
agent { label 'maven' }
steps {
script {
echo "This is where we publish to Nexus"
module_Artifact.publish()
}
}
}
}
post {
always {
script {
module_Notification.sendEmail(currentBuild.result)
}
}
}
}
}
When we break this down into readable steps, it does the following things:
- Builds and runs the unit tests with Maven
- Gathers the test results to report on the status
- Publishes the artefact to Nexus
- Sends an email about the status of this run
If you want to read more about this declarative style of Jenkins Pipelines, check out the manual! It is my go-to reference for anything related to the syntax and the built-in steps.
Creating the test
I’m adhering to the standard of appending the word test to the name of the class under test, so I created the file test/defaultPipelineTest.groovy.
The scaffold-file would look something like this:
import groovy.testSupport.PipelineSpockTestBase
class defaultPipelineTest extends PipelineSpockTestBase {
def script
def setup() {
script = loadScript('vars/defaultPipeline.groovy')
}
def cleanup() {
printCallStack()
}
void 'Happy flow'() {
when:
script.call([:])
then:
assertJobStatusSuccess()
}
}
This is the absolute minimal version of our test, and I’ll explain what is in there.
First off, we create a variable to hold our runnable pipeline (script). We then see a setup method which is run before each test (feature method). (Read more about Spock terminology here) The cleanup method is run after each test. Finally, there’s the void 'Happy flow'() {} which is the actual test (or Feature Method as Spock likes to call it).
This test uses the when -> then style of testing, and just calls the pipeline with a default set of parameters and validates the outcome.
Please note it actually doesn’t test for anything apart from the fact that it exits successfully. None of the previously mentioned ‘features’ are validated.
Now, it’s time to run the test!
gradle clean check is all you need, and you’ll find it fails spectacularly. 
No signature of method: defaultPipeline.module_Maven() is applicable for argument types: (String) values: [clean verify] groovy.lang.MissingMethodException: No signature of method: defaultPipeline.module_Maven() is applicable for argument types: (String) values: [clean verify] at com.lesfurets.jenkins.unit.PipelineTestHelper$_closure4.doCall(PipelineTestHelper.groovy:182) .... at defaultPipeline.call(defaultPipeline.groovy:5) at com.lesfurets.jenkins.unit.PipelineTestHelper.callMethod(PipelineTestHelper.groovy:156) at com.lesfurets.jenkins.unit.PipelineTestHelper$_closure3.doCall(PipelineTestHelper.groovy:143) at minimalTest.Happy flow(minimalTest.groovy:17)
This is the slightly convoluted way of Spock telling you it didn’t know how to process line 5 of defaultPipeline.groovy: which is the call to our own module_Maven. We’ll have to add this ourselves then!
Mocking our own code
The Jenkins Pipeline Unit framework has very convenient helper methods to teach the framework about methods which do not come built into the framework. Our module_Maven is, of course, one of those so we need to add the signature of this method to our framework at runtime.
There actually are two very specific styles for this, I’ll discuss them both here.
The module_Maven has a call method, which enables us to call it implicitly by just calling the complete step.
Ergo: module_Maven.call('clean test') is functionally equal to module_Maven('clean test').
This difference only matters for the way our tests have to handle them.
Implicit call
Our frameworks enable us to both register steps as allowed methods and define behaviour for them. And as a bonus, it’s even possible to verify calls to our mocked steps and check any parameters used in the call itself!
As the call to module_Maven will be used in virtually all tests for our pipeline, it’s best to define the Mock for it only once and at a global level. We’ll add a variable to the Test class for this. We’ll also need to instantiate this Mock for every test, so we need to do so inside our setup() method. As we have multiple Mocks for this pipeline, I’ve created a new helper-method called registerMocks() to keep the setup() method clean and simple.
Finally, after initializing our Mock we need to register the allowed method module_Maven in the test framework, for which we can use the method registerAllowedMethod on the helper object. This method takes three parameters:
- The string containing the name of the allowed method
- An array of classes of the parameters the allowed method takes
- The object containing the Mock which is to be invoked during our tests.
When combined, it gives us additional code like below:
def mavenMock
...
def registerMocks() {
mavenMock = Mock(Closure)
helper.registerAllowedMethod('module_Maven', [String.class], mavenMock)
}
Our setup method looks like this:
def setup() {
registerMocks()
script = loadScript('vars/defaultPipeline.groovy')
}
So, to recap: an implicit call requires:
- A (global) variable
- Initialisation of the Mock
- Register the allowed method on the helper
Explicit call
Our pipeline also has an explicit call to a Custom Pipeline Step, which is module_Artifact.publish().
This requires a slightly different approach:
- Import the mocked class
- A (global) variable
- Initialisation of the Mock
- Register the Mock in the framework
In code, that would look something like this:
import module_Artifact
...
def artifactMock
...
def registerMocks() {
...
artifactMock = Mock(module_Artifact)
binding.setVariable('module_Artifact', artifactMock)
}
As you can see, we only need to create a Mock for the entire Module we’re using, instead of adding each and every method independently. Also, we do not register a Method on the helper, but we set a Variable on the binding object which is provided by our frameworks. The method takes two parameters: the key (name of the module) and the value (the Mock).
Mocking steps from plugins
Our pipeline also uses steps from Jenkins plugins which do not come by default with a Jenkins installation, one of those is the step junit.
Mocking one of those is much like mocking an implicitly called Custom Step.
I like to separate the steps provided by plugins from our own steps by creating a new helper-method: registerPluginMethods()
def registerPluginMethods() {
// Junit
// https://plugins.jenkins.io/junit
helper.registerAllowedMethod('junit', [HashMap.class], null)
}
The setup() method now looks like this:
def setup() {
registerMocks()
registerPluginMethods()
script = loadScript('vars/defaultPipeline.groovy')
}
Putting it all together
We only need one more Custom Step to be Mocked: module_Notification.
When we put all the previously discussed steps together, we end up with a (very minimal) test-script like this:
import groovy.testSupport.PipelineSpockTestBase
import module_Artifact
import module_Notification
class minimalTest extends PipelineSpockTestBase {
def script
def mavenMock
def artifactMock
def notificationMock
def setup() {
registerMocks()
registerPluginMethods()
script = loadScript('vars/defaultPipeline.groovy')
}
def cleanup() {
printCallStack()
}
void 'Happy flow'() {
when:
script.call([:])
then:
assertJobStatusSuccess()
}
def registerPluginMethods() {
// Junit
// https://plugins.jenkins.io/junit
helper.registerAllowedMethod('junit', [HashMap.class], null)
}
def registerMocks() {
mavenMock = Mock(Closure)
helper.registerAllowedMethod('module_Maven', [String.class], mavenMock)
artifactMock = Mock(module_Artifact)
binding.setVariable('module_Artifact', artifactMock)
notificationMock = Mock(module_Notification)
binding.setVariable('module_Notification', notificationMock)
}
}
When we run this test, you’ll end up with something like this in the logs:
>>>>>> pipeline call stack -------------------------------------------------
defaultPipeline.call({})
defaultPipeline.pipeline(groovy.lang.Closure)
defaultPipeline.agent(groovy.lang.Closure)
defaultPipeline.stages(groovy.lang.Closure)
defaultPipeline.stage(Build and Unit test, groovy.lang.Closure)
defaultPipeline.agent(groovy.lang.Closure)
defaultPipeline.label(maven)
defaultPipeline.steps(groovy.lang.Closure)
defaultPipeline.script(groovy.lang.Closure)
defaultPipeline.module_Maven(clean verify)
defaultPipeline.post(groovy.lang.Closure)
defaultPipeline.always(groovy.lang.Closure)
defaultPipeline.junit({testResults=**/target/surefire-reports/*.xml, allowEmptyResults=false})
defaultPipeline.stage(Publish to Nexus, groovy.lang.Closure)
defaultPipeline.agent(groovy.lang.Closure)
defaultPipeline.label(maven)
defaultPipeline.steps(groovy.lang.Closure)
defaultPipeline.script(groovy.lang.Closure)
defaultPipeline.echo(This is where we publish to Nexus)
defaultPipeline.post(groovy.lang.Closure)
defaultPipeline.always(groovy.lang.Closure)
defaultPipeline.script(groovy.lang.Closure)
Conclusion
Now that we have created a full pipeline and a test containing a very simple way of Mocking and validating the pipeline to just run, it would be much nicer to actually verify that what is happening, actually is what we want. I’ll elaborate more on this in the coming weeks!
