Home
This plugin allows you to maintain jenkins configurations in source control and apply them to a Jenkins server via gradle. Jobs and views can be stored as straight xml files, xml strings, markup builder closures or as jenkins-job-dsl. Job templates can be defined that can then be manipulated such that multiple jobs can be generated off of a single template definition.
Powerful use of this plugin requires intimate knowledge of a job's config.xml. A standard use case would be to configure a job in Jenkins, download the config.xml and store it in source control. The xml could then be manipulated at runtime to customize the job based on values from the gradle configuration, configure multiple jobs and/or apply them to multiple servers. Additionally, purely programmatic configuration of a job is also possible using dsl.
See the Gradle plugin portal page for this plugin.
Note that with versions 1.2+, the plugin id has changed to a qualified form to comply with the new Gradle plugin portal (i.e. from 'jenkins' to 'com.terrafolio.jenkins').
This first example shows a very simple case where the jenkins job is stored as an xml file and applied to the server as is.
apply plugin: 'com.terrafolio.jenkins'
jenkins {
servers {
testing {
url 'http://jenkins.somewhere.com:8080'
secure true // optional
username "testuser" // optional
password "testpass" // optional
}
}
defaultServer servers.testing // optional
jobs {
test {
server servers.testing
definition {
name "Build ${project.name}" //optional
xml file('config.xml')
}
}
}
}The following gradle file demonstrates the use of templates with xml manipulation. It uses a single template xml file (downloaded straight from <JOB_URL>/config.xml) and then generates two new jobs - one for the master branch of a git repository and one for the develop branch. For each, it uses a closure to override the template to 1) disable the job, 2) set a custom workspace, 3) set the repository url, and 4) set the branch name in the SCM configuration.
apply plugin: 'com.terrafolio.jenkins'
jenkins {
servers {
testing {
url 'http://jenkins.somewhere.com:8080'
secure true // optional
username "testuser" // optional
password "testpass" // optional
}
}
templates {
build {
xml file('build-template.xml')
}
}
defaultServer servers.testing // optional
jobs {
[ 'master', 'develop' ].each { branchName ->
"build_${branchName}" {
server servers.testing // optional
definition {
name "Build ${project.name} (${branchName})"
xml templates.build.override { projectXml ->
projectXml.disabled = 'true'
projectXml.customWorkspace = "/build/${branchName}/${project.name}"
projectXml.scm.userRemoteConfigs.'hudson.plugins.git.UserRemoteConfig'.url = "git@gitserver:${project.name}.git"
projectXml.scm.branches.replaceNode { node ->
branches() {
'hudson.plugins.git.BranchSpec'() {
name([:], "*/${branchName}")
}
}
}
}
}
}
}
}
}The following example shows the use of server-specific configurations. Note that the closure added to the server directive is the same as added for job.definition.
apply plugin: 'com.terrafolio.jenkins'
jenkins {
servers {
test1 { url 'http://jenkins1.somewhere.com' }
test2 { url 'http://jenkins2.somewhere.com' }
}
jobs {
build {
server servers.test1, {
name 'Build for test1' // optional
xml override { projectXml ->
projectXml.description = 'This build is for test1'
}
}
server servers.test2, {
name 'Build for test2' // optional
xml override { projectXml ->
projectXml.description = 'This build is for test2'
}
}
definition {
name 'Build job'
xml file('config.xml')
}
}
}
}Note that server-specific configurations are always applied last. For instance, if you had a job that used a template and then overrode the template, those overrides would be evaluated first (regardless of order in the script) and the server-specific overrides would occur last.
Manipulating XML is an easy way to make simple modifications to existing job definitions, but you can also configure your jobs from scratch using jenkins-job-dsl (https://github.com/jenkinsci/job-dsl-plugin/wiki/Job-DSL-Commands). This is done with the dsl { } configuration on the job object. The following example shows a new job being configured with dsl:
apply plugin: 'com.terrafolio.jenkins'
jenkins {
servers {
testing {
url 'http://jenkins.somewhere.com:8080'
secure false
}
}
jobs {
build_job {
server servers.testing
dsl {
customWorkspace '/workspaces/build'
scm {
git('git@git.somewhere.com/build_repo')
}
triggers {
scm '*/5 * * * *'
}
steps {
gradle 'clean build'
}
publishers {
archiveJunit 'build/test-results/*.xml'
archiveArtifacts 'build/libs/*.jar'
}
}
}
}
}The above example creates a brand new job and sets up the workspace, the scm configuration, the build steps and some publishers.
Similar to XML manipulation, dsl-defined jobs can be templated:
apply plugin: 'com.terrafolio.jenkins'
jenkins {
servers {
testing {
url 'http://jenkins.somewhere.com:8080'
secure false
}
}
templates {
build_template {
dsl {
triggers {
scm '*/5 * * * *'
}
steps {
gradle 'clean build'
}
publishers {
archiveJunit 'build/test-results/*.xml'
archiveArtifacts 'build/libs/*.jar'
}
}
}
}
jobs {
for (i in 1..3) {
"build${i}" {
server servers.testing
dsl {
name "Build Job #${i}"
using 'build_template'
customWorkspace '/workspaces/build${i}'
scm {
git('git@git.somewhere.com/build_repo_${i}')
}
}
}
}
}
}The above creates three jobs based on the "build_template" template and configures the name, git repository and workspace for each.
There is also a mutliple job form where you can store all of your job information in separate dsl scripts and load them from the file system via a dsl method on the root jenkins object. So if you already have a lot of jenkins-job-dsl defined, you can incorporate that work directly into the gradle build.
apply plugin: 'com.terrafolio.jenkins'
jenkins {
servers {
testing {
url 'http://jenkins.somewhere.com:8080'
secure true // optional
username "testuser" // optional
password "testpass" // optional
}
}
defaultServer servers.testing // required
dsl fileTree('jenkins').include('*.dsl')
}The above example finds files in the "jenkins" sub-directory with a "dsl" extension and configures the jobs defined in each. The dsl files would be in the format of standard jenkins-job-dsl:
// DSL File Example
job {
name 'Build Job'
customWorkspace '/workspaces/build'
git 'git@git.somewhere.com/build_repo'
steps {
gradle 'clean build'
}
publishers {
archiveJunit 'build/test-results/*.xml'
archiveArtifacts 'build/libs/*.jar'
}
}Note that when loading job definitions from dsl files, the job name is the name specified by the dsl (e.g. after applying the dsl file example above, the job could be accessed as jenkins.jobs.'Build Job'). Also, there is no way in jenkins-job-dsl to specify the server for each job, so a defaultServer must be provided when using the multiple job dsl form.
Similarly, multiple jobs can be defined with standard jenkins-job-dsl using inline dsl rather than separate files:
jenkins {
servers {
testing {
url 'http://jenkins.somewhere.com:8080'
secure true // optional
username "testuser" // optional
password "testpass" // optional
}
}
defaultServer servers.testing // required
dsl {
for (i in 1..3) {
job {
name "Build Job ${i}"
customWorkspace "/workspaces/build${i}"
scm {
git("git@git.somewhere.com/build_repo_${i}")
}
steps {
gradle 'clean build'
}
publishers {
archiveJunit 'build/test-results/*.xml'
archiveArtifacts 'build/libs/*.jar'
}
}
}
}
}Views can also be defined and deployed to jenkins using jenkins-job-dsl. See https://github.com/jenkinsci/job-dsl-plugin/wiki/View-Reference for view dsl reference. For example:
apply plugin: 'com.terrafolio.jenkins'
jenkins {
servers {
testing {
url 'http://jenkins.somewhere.com:8080'
secure false
}
}
defaultServer servers.testing // required
dsl {
for (i in 1..3) {
job {
name "Build Job ${i}"
customWorkspace "/workspaces/build${i}"
scm {
git("git@git.somewhere.com/build_repo_${i}")
}
steps {
gradle 'clean build'
}
publishers {
archiveJunit 'build/test-results/*.xml'
archiveArtifacts 'build/libs/*.jar'
}
}
}
}
views {
"Odd jobs" {
server servers.testing // Optional
dsl {
jobs {
names(
// Build Job 1, Build Job 3
(1..jenkins.jobs.size())
.findAll { it%2 != 0 }
.collect { "Build Job ${it}" } as String[]
)
}
columns {
name()
buildButton()
}
}
}
"Even jobs" {
server servers.testing // Optional
dsl {
jobs {
names(
// Build Job 2
(1..jenkins.jobs.size())
.findAll { it%2 == 0 }
.collect { "Build Job ${it}" } as String[]
)
}
columns {
name()
buildButton()
}
}
}
}
}Views can also be configured via dsl files or as part of the multiple job dsl block using standard jenkins-job-dsl much like jobs.
// DSL file example
view {
name "Even jobs"
jobs {
name "Build Job 2"
}
columns {
name()
buildButton()
}
}The following conventions are added by this plugin:
-
jenkins - The main configuration closure contains servers, templates, jobs, views and (optionally) a defaultServer definition.
-
servers - Definitions of jenkins servers where jobs can be applied. Each named server can define several fields:
- url - The url to the jenkins instance.
- username - The username to use (must have admin privileges). (Optional)
- password - The password to use. (Optional)
- secure - Whether or not the server is secured (requiring username and password). If username and password are not defined, and secure is set to true, they will be prompted for on the console. If no console is available, an exception will be thrown. In the event that the secure field is set to false, empty values for username and password are allowed and no prompting will occur. (Optional, default = true)
-
defaultServer - The default server to use when a job does not specify a server. If a defaultServer is not defined, then each job must specify the server it should be applied to. (Optional)
-
templates - Definitions of jobs that can be used as templates for concrete jobs. Each named template defines one field:
- xml - The config.xml to use as a template for defining jobs. This field can accept a String, a File, or a Groovy MarkupBuilder closure.
- dsl - Jenkins-job-dsl closure that defines the job template. This field can accept either a dsl File or inline dsl. Inline dsl here should conform to the contents of a "job { }" closure in jenkins-job-dsl (https://github.com/jenkinsci/job-dsl-plugin/wiki/Job-DSL-Commands).
-
dsl - Jenkins-job-dsl specifying one or more jobs or views (https://github.com/jenkinsci/job-dsl-plugin/wiki/Job-DSL-Commands). This can accept a FileCollection or a closure with inline dsl.
-
jobs - Definitions of concrete jobs. Each named job defines several fields:
- server - The server the job should be applied to and the job definition. If the server is not configured it will use the server defined by defaultServer. Multiple "server" declarations causes the job to be applied to multiple servers. You can also add an xml manipulation closure argument to do server-specific configuration. (Optional)
-
definition - The definition of the actual job. This closure defines the following fields:
- name - The name of the job on the Jenkins server. This defaults to the name of the job. (Optional)
- xml - The config.xml that defines the job. This field accepts a String, a File, or a Groovy MarkupBuilder closure. Additionally, if a template is defined, you can use the override method on the template which accepts a closure to manipulate the content. The GPathResult from an XMLSlurper is passed to the closure for manipulation. See http://groovy.codehaus.org/Updating+XML+with+XmlSlurper for info on using XMLSlurper GPathResults to manipulate xml.
- dsl - Jenkins-job-dsl closure that defines the job. This field can accept either a dsl File or inline dsl. Inline dsl here should conform to the contents of a "job { }" closure in jenkins-job-dsl (https://github.com/jenkinsci/job-dsl-plugin/wiki/Job-DSL-Commands).
- type - When specifying the job definition via dsl, this represents the job type. Valid types are Freeform (default), BuildFlow, Multijob and Maven. (Optional)
-
views - Definitions of concrete views. Each named view defines several fields:
- server - The server the view should be applied to and the job definition. If the server is not configured it will use the server defined by defaultServer. Multiple "server" declarations causes the view to be applied to multiple servers. (Optional)
- xml - The config.xml that defines the view. This field accepts a String or a File.
- dsl - Jenkins-job-dsl closure that defines the view. This field can accept either a dsl File or inline dsl. Inline dsl here should conform to the contents of a "view { }" closure in jenkins-job-dsl (https://github.com/jenkinsci/job-dsl-plugin/wiki/View-Reference).
- type - When specifying the view definition via dsl, this represents the view type. Valid view types are ListView (default) and BuildPipelineView. (Optional)
-
The plugin applies the following tasks to the project:
- updateJenkinsItems - Creates or updates jobs/views on the server(s). This task operates on whatever jobs are specified by the job or server filters. If no filters are applied, it operates on all jobs.
- updateJenkinsJobs - (Deprecated) Same as updateJenkinsItems.
- deleteJenkinsItems - Deletes jobs/views from the server(s). This task operates on whatever jobs are specified by the job or server filters. If no filters are applied, it operates on all jobs.
- deleteJenkinsJobs - (Deprecated) Same as deleteJenkinsItems.
- dumpJenkinsItems - Writes job/view configurations from the local model to files under buildDir ("build" by default). Items are written to "${buildDir}/${server}/[jobs|views]/${itemname}.xml". This task operates on whatever jobs/views are specified by the job or server filters. If no filters are applied, it operates on all jobs. By default, this task dumps xml in a human-friendly form, but this format may not import well into jenkins due to line breaks and other white space (if you attempt to import it manually). Setting "prettyPrint = false" on this task causes the xml to be dumped in a form that should import well, but is decidedly not friendly to humans.
- dumpJenkinsJobs - (Deprecated) Same as dumpJenkinsItems
- dumpRemoteJenkinsItems - Like dumpJenkinsItems, but dumps configurations from the remote server(s). Items are written to buildDir/remotes (e.g. "build/remotes) following the pattern described for dumpJenkinsItems above.
- retireJenkinsItems - Deletes jobs/views specified for retirement. This task only operates on the jobs specified.
- retireJenkinsJobs - (Deprecated) Same as retireJenkinsItems.
- validateJenkinsItems - Validates that the jobs on the server match the jobs configured in the gradle configuration. Note that this an xml comparison that checks if the two documents have the same elements, attributes and content. There are a number of situations where the jobs may function exactly the same, but the xml may differ enough for this task to show a difference.
- validateJenkinsJobs - (Deprecated) Same as validateJenkinsItems.
The plugin also applies the following task rules:
- updateJenkinsItems<server> - Like updateJenkinsItems, but only updates items on the server specified.
- deleteJenkinsItems<server> - Like deleteJenkinsItems, but only deletes items on the server specified.
- dumpJenkinsItems<server> - Like dumpJenkinsItems, but only dumps items for the server specified.
- dumpRemoteJenkinsItems<server> - Like dumpRemoteJenkinsItems, but only dumps items from the server specified.
- validateJenkinsItems<server> - Like validateJenkinsItems, but only validates against the server specified.
By default, the plugin will attempt to determine if an item is different from the configuration that would be uploaded. It does this by testing if the item's xml is "similar" to the live configuration xml (i.e. it has the same elements, attributes and values). If an item has not been edited in any way after it has been uploaded, subsequent runs of the update task will find no differences and skip the update for that item. However, if changes are being made to items through the Jenkins gui after it has been uploaded, Jenkins tends to make a number of changes to the xml that will likely show up as a difference (even if the job is functionally the same) and the plugin will update the item anyways.
This up-to-date check can be overridden by setting the project property forceJenkinsJobsUpdate to 'true'. This can be done by setting the property on the command line (i.e. "-PforceJenkinsJobsUpdate=true") or by setting this as a project property in your build script (i.e. "ext.forceJenkinsJobsUpdate = 'true'"). With this property set, all items will be updated regardless of whether they have been changed or not.
By default, the validateJenkinsItems task causes a build failure whenever a difference is found. If, however, one wants to run this task for informational purposes, but not have it fail on differences, this behavior can be overridden:
validateJenkinsItems.failOnDifference = falseAt normal log levels, the validateJenkinsItems task only outputs information about any items that do not match the configuration. For additional information, including details about each difference as well as information on which items do match the configuration, use the "info" log setting (i.e. the "--info" command line switch).
As your Jenkins configuration evolves, you may find the need to retire large numbers of old items. A convenience task (retireJenkinsItems) is provided to facilitate this programmatically. Example:
retireJenkinsItems {
delete(jenkins.servers.server1, "Build job1")
delete(jenkins.jobs.job2)
}The delete method has two forms. The first accepts a server definition and the name of the item in Jenkins as the arguments. The second form takes an item defined in the jenkins configuration. The first form is most likely to be used as you would probably remove your job from the configuration prior to running this task. Retirement of jobs can be easily hooked into updates by setting a dependency on updateJenkinsItems.
Views can also be retired:
retireJenkinsItems {
deleteView(jenkins.servers.server1, "Build View")
deleteView(jenkins.views."Build View")
}Note that retireJenkinsItems is only a convenience task and deleting individual jobs can be done in a more general purpose fashion if necessary by overriding the DeleteJenkinsItemsTask task. Example:
import com.terrafolio.gradle.plugins.jenkins.tasks.DeleteJenkinsItemsTask
...
task deleteSomeJobs(type: DeleteJenkinsItemsTask) { task ->
jenkins.servers.each { server ->
task.delete(server, "Some job")
}
}You can filter which jobs are operated on by setting the "jenkinsJobFilter" property on the build to a regex matching the item name. The most likely case is that you will want to do this from the command line (e.g. -PjenkinsJobFilter="build.*"). The name the regex matches against is the name of the item in gradle, not the name of the project in Jenkins. For instance:
jobs {
build_lib_1 {
server servers.testing
definition {
name "Build Library 1"
xml file("lib1-config.xml")
}
}
build_lib_2 {
...
}
build_war_1 {
...
}
deploy_war_1 {
...
}
}If you used -PjenkinsJobFilter='.*war_1', it would match jobs build_war_1 and deploy_war_1 and only apply the changes to those jobs in Jenkins. On the other hand, if you specified the regex as 'build_.*', it would match build_lib_1, build_lib_2, and build_war_1.
In addition to the task rules allowing you to perform operations on a per-server basis, you can also filter which servers are operated on by setting the "jenkinsServerFilter" property on the build to a regex matching the server name. The most likely case is that you will want to do this from the command line (e.g. -PjenkinsServerFilter="test.*"). For instance:
servers {
test_1 {
...
}
test_2 {
...
}
qa {
...
}
prod {
...
}
}If you used -PjenkinsServerFilter='test.*', it would apply any jobs set up to run against test_1 or test_2.
See release description.
- Merge pull requests and upgrade version of job-dsl-core to v1.3.3.
See release description.
- Merge pull requests and upgrade versions of various dependencies.
See release description.
- v1.3.0 was published to the wrong maven group id. This release publishes to the "com.terrafolio" group id to be consistent with past releases.
See release description.
- Updated versions of dependency libraries including updating job-dsl-core to 1.34.
See release description.
- Fixed minor regression in template support where xml file and closure form were no longer working.
See release description.
- Support for new Gradle plugin portal. This release changes the plugin id from 'jenkins' to 'com.terrafolio.jenkins' to comply with the new id naming scheme.
- Refactor of class hierarchy to eliminate some anti-patterns and awkward relationships.
- Issue #38: Support for git credentials
See release description.
- Issue #27: Refactor task names to use JenkinsItems
- Issue #34: Restful calls do not use full url
- Added dumpRemoteJenkinsItems task to allow comparison of remote jobs to the local model
See release description.
- Issue #9: Add support for adding views to jenkins
- Issue #20: Add support for jenkins-job-dsl (https://github.com/jenkinsci/job-dsl-plugin/wiki/Job-DSL-Commands) to provide DSL for convenience (Thanks to @sgoings for proposing this idea!)
- Issue #5: Check existing jobs for changes before update
- Issue #18: Problem with creating jobs on Jenkins 1.535 (Note that this version is required if you're using 1.535)
- Pull Request #16: Update for jdk7 compatibility
- Issue #15: Make job name convention for job definition name
- Issue #14: DumpJenkinsJobs does not dump server-specific configurations
- Issue #13: white space in dumped job files cannot be imported
- Issue #12: config style closure on job definition overwrites existing definition
- Issue #11: Password prompting occurs for dumpJenkinsJobs
- Test refactoring and issue #10: adding support for composing update jobs
- Issue 4: Add ability to dump job xml
- Issue 6: Add support for retiring jobs
- Issue 7: Move server credential prompting to beginning of task
- Issue 8: Remove asm dependency from http-builder
- Add support for filtering jobs to operate against (instead of all jobs all the time).
- Add support for filtering servers to operate against (instead of all servers all the time).
- Add support for per-server configuration such that slightly different configurations can be deployed to different servers for the same job.
Suggestions, contributions and/or bug reports are welcome. Please log any as a pull request or an issue in github (https://github.com/ghale/gradle-jenkins-plugin/issues).