Jenkins, a widely-used open-source automation server, offers a powerful REST API that enables programmatic interaction with Jenkins.

This blog post outlines the process of creating a Jenkinsfile that utilizes the Jenkins REST API to set up a new Jenkins job based on a config.xml file. This integration was developed in conjunction with GitHub and Backstage to automate the creation of Jenkins jobs for multiple repositories within a GitHub organization. By using an “Organization Folder” Jenkins job, the pipeline is triggered for any existing or new repository within the organization, as long as a Jenkinsfile is present.

The primary goal of this integration is to streamline the creation of new Jenkins jobs for repositories generated by Backstage. Depending on the programming framework used, an appropriate Jenkins job is created, based on a template, to build the code, generate a Docker image, and deploy it to a Kubernetes engine. While scaffolding in Backstage was an option, the Jenkinsfile-based solution proved to be more straightforward and efficient.

With this context in mind, the following steps detail how to create a Jenkins job from within a Jenkinsfile.

1. Create an API token to call Jenkins REST API:

Before you can interact with the Jenkins REST API, you need to create an API token. This token will be used to authenticate your API calls. To create an API token, log in to your Jenkins instance, click on your username in the top right corner, and then click on “Configure.” In the “API Token” section, click on “Add new Token,” provide a name for the token, and click on “Generate.” Save the generated token somewhere safe, as you will not be able to see it again.

2. Create credentials with a username and password for use in the Jenkins pipeline:

In addition to the API token, you will also need to create credentials with a username and password in Jenkins. These credentials will be used to authenticate your API calls within the Jenkins pipeline. To create the credentials, follow these steps:

• Go to the Jenkins home page and click on “Credentials” in the left-hand menu.
• Click on the “System” tab and then click on the “Global credentials (unrestricted)” link.
• Click on the “Add Credentials” button and select “Username with password” as the credential type.
• Enter a meaningful name for the credential, the username, and the password.
• Click “OK” to save the new credential.

1. Update your Jenkinsfile to call the Jenkins REST API:

Now that you have your API token and credentials in place, you can update your Jenkinsfile to call the Jenkins REST API. To do this, you can use the withCredentials block in combination with a script block in your Jenkinsfile. Here’s an example:

pipeline {
    agent any

    stages {
        stage('Create Job') {
            steps {
                script {
                    withCredentials([usernamePassword(credentialsId: 'my-credential', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) {
                        sh "curl -X POST -u ${USERNAME}:${PASSWORD} --header \"Accept: application/json\" --data-binary @path/to/config.xml ${env.JENKINS_URL}/createItem?name=MyNewJob"
                    }
                }
            }
        }
    }
}

In this example, the withCredentials block is enclosed within a script block, which allows the pipeline to access the environment variables set by the usernameVariable and passwordVariable options. The curl command is updated to include the –header option, which sets the Accept header to application/json, ensuring that the response from the Jenkins REST API is in JSON format.

By following these steps, you can create a Jenkinsfile that calls the Jenkins REST API to set up a new Jenkins job based on a config.xml file. This will enable you to automate and streamline your Jenkins job creation process, making your software development pipeline more efficient and easier to manage.