Some time ago (probably some years) I wrote a wiki article “IBM i and Continuous Integration”. I was kind of successful with my adventure in the realm of CI and DevOps to some degree. I could install the CI server but had some trouble with running the commands on IBM i.
Some years and software versions later I am giving it a try again. I think CI is an important part in software development and not so hard to implement (at least when it comes to the basics).
So what is the goal?
Continuous Integration!
But what is “Continuous Integration”? Again I will refer to the Wikipedia article Continous integration. I think the main point is this:
… the practice of merging all developers’ working copies to a shared mainline several times a day that triggers automated build with testing …
And today this merging can be done even easier with the version control systems of today which can trigger a build automatically after a successful merge.
Why CI even if you are a single developer in your shop?
Having a build server can be handy for many reasons:
- Build server can be setup to have the same environment as the production environment. The development environment often differs from the production environment.
- Every build is done in exactly the same way.
- Builds can be triggered regardless of the developer machine.
- Release builds
- … and there are many many more
Here some quotes from the net:
The CI build can also be thought of as your “release” build. The environment should be stable, and unaffected by whatever development gizmo you just add to your machine. It should allow you to always reproduce a build. This can be valuable if you add a new dependency to your project, and forget to setup the release build environment to take that into account. — Is Continuous Integration important for a solo developer?
Likewise, if you are building libs that are used by multiple projects then CI will make sure they work with ALL of the projects rather than just the one that you’re working with right now… — Is Continuous Integration important for a solo developer?
So … Continuous integration is for everyone!
Task
We will take a RPG project with the source located in a Git repository and will build it with
Jenkins on an IBM i server. The build will be executed with the tool make
. We will see what
Jenkins and its’ plugins can bring to the table.
Build Server
Nothing has changed here. I am still a big fan of Jenkins. Jenkins is written in Java and runs very well on IBM i without any modifications. You should install Java 11 on your IBM i either via yum or as a licensed product.
We can test the java installation with executing java -version
on the shell. Your output in a
shell should look something like this:
1 | -bash-5.1$ java -version |
The important part is the version “11.0.11-ea”. It should at least state version 11. Java in general is very backwards compatible. A more recent version than 11 should work without any problems.
Installation
Jenkins can be downloaded from the Jenkins website.
The installation is pretty straight forward. Jenkins is packaged as a war file but also supports executing the file as a normal Java application.
The easiest way to start Jenkins is from a shell (f. e. bash via a SSH connection):
1 | java -jar jenkins.war |
The Jenkins project has an extensive documentation. The website provides a page Installing Jenkins. The WAR file section is probably the documented installation platform which maps best to IBM i PASE (AIX).
The Jenkins runtime can be configured via command line parameters and Java system properties. A list of supported command line parameters (f. e. HTTP port) can also be found here. The extensive list of Java system properties can be found here.
By default all the data will be stored in the .jenkins
folder in the home directory of the
current user and the default port is 8080.
This might not be convenient for your environment and we can change that to some other values.
1 | export JENKINS_HOME=/var/local/lib/jenkins |
Now Jenkins uses port 5000 and stores its data in the folder /var/local/lib/jenkins
.
Note: You may need to add the Java system property “java.net.preferIPv4Stack=true” when you
see Jetty choking out some weird network exception. So your command line then looks like
java -Djava.net.preferIPv4Stack=true -jar jenkins.war --httpPort=5000
.
Configuration
The path to the tools used in the build jobs need to either be specified in Jenkins or need to be in the PATH environment variable.
The PATH environment variable can be configured in Jenkins management section.
1 | Dashboard ⇒ Manage Jenkins ⇒ Configure System ⇒ Global Properties |
The location of some essential tools can be configured extra in
1 | Dashboard ⇒ Manage Jenkins ⇒ Global Tool Configuration |
If you don’t have git
in the PATH you can set it here to /QOpenSys/pkgs/bin/git
.
Build Job
A Build Job defines what is executed and also where and when. It defines the execution environment. In Jenkins you can configure a job in different ways. I will cover “Freestyle Project” and “Pipeline”. Freestyle projects consist of predefined build steps which can be concated to a workflow. Pipeline projects are mainly defined by a pipeline script which is written in a specific DSL. Pipeline scripts can be written in two different styles: scripted and declarative. In the declarative style you use mainly predefined functions and don’t script as much yourself (though you also have the option to script in the declarative pipeline). The Pipeline project will be covered in more detail in another post. Here I will focus on the “Freestyle” project.
Freestyle Project
Build Jobs can be configured in Jenkins in Dashboard ⇒ New Item
. Configure the job as a
“Freestyle Project”. Freestyle projects are configured in the UI with build steps and configuration
options contributed by other Jenkin plugins.
The workflow of a “freestyle” build consists of steps. Steps can either be added as “Build Steps” or “Post-Build Steps” by using the corresponding dropdown button in the configuration of the build job.
Git
In a freestyle project there is a predefined option to check out the source code from a source code repository. Here we can enter our URL for the Git repository.
Compile
This project uses make
for building the project. We first execute the purge
target to have
a clean build environment and then the all
target. We specify the target library as a parameter
of the gmake command, see BIN_LIB
.
1 | gmake BIN_LIB=MIHAELSTMP purge all |
Note: I am using gmake
which can be installed via yum
which is the GNU version of make
.
We add a build step “Execute shell”.
What happens on a failed compile?
Easy. gmake
will return a non zero exit value if it cannot execute a target successfully. By
default the shell step will mark the build as failed when it gets a non zero exit value from
the executed shell command. So on a failed compile the build will automatically be marked as
failed.
Open Tasks
In the old wiki article I mentioned the Task Scanner Plugin for scanning the source code for specific keywords which mark open tasks. That was a couple of years ago. In the meantime many plugins have been merged into the Warnings Next Generation plugin which also now includes the feature set of the Task Scanner plugin.
In a freestyle project the open task scanning can be added as a post-build action by selecting “Record compiler warnings and static analysis results”.
The result can be found by selecting the menu entry “Open Tasks” on the main page of the executed build.
Notification
Having an automated build is pretty cool. But what about the results. In most cases we would want to know the result of the build. But we don’t want to stare at the screen and wait for the result. We want to be informed by Jenkins when the build has finished.
Jenkins has many plugins which cover the different ways of communicating the result of a build.
But the most simple way is to get an email. We can use the default email plugin or use the extended
email plugin. Both plugins can be configured under Dashboard ⇒ Manage Jenkins ⇒ Configure System
.
But just installing and configuring the email plugin is not enough. That will not trigger an email on a finished build. We need to add a Post-Build action step like “E-mail Notification” or “Editable Email Notification”.
Starting Jenkins
At the end I would like to introduce you to another option of starting Jenkins on IBM i. The project Service Commander from Jesse Gorzinski does a wonderful job of managing services and keeps configuring the environment of those services simple. Especially the Java and Node.js applications make heavy use of environment variables and that is one area where Service Commander really shines.
I don’t want to go into the details of how to install and use Service Commander because Jesse already did a great job of that on the project Github site. Just read the README file. But I want to share my start script with you.
1 | name: Jenkins |
The beauty of this is that you can define your environment without any hassle and still integrate with the traditional management of jobs on the IBM i server. This Jenkins instance will have JENKINS as its job name and thus can be easily found.
Kudos to Jesse!
Wrap up
With a couple of steps we can have quite a smooth development lifecycle which automates many of those tedious steps and let us play it safe with source code scanning and unit testing … in a team or as a lone wolf.
There are still some areas which we haven’t covered in detail: unit testing and documentation. Perhaps this will be covered in a future post.
Jenkins also supports triggering a build by a HTTP request. But as this post is already very lengthy this topic may be covered in another post.
Happy building!
Mihael