Open Source on IBM i

I can package it all by myself

... and also provide those packages to others.

But you probably know this expression much better as "I can read it all by myself". I have read those books from Dr. Seuss very often and with pleasure. Some were real tongue twisters.

But what I really like about the books is that they inspire you to do things yourself. To not wait for others to do the work. And this is a great problem in the IBM i community as many are waiting for IBM to do the work. But when we look at other platforms it is not the big companies who started the contributions to their community. It is the small developer in his spare time. Getting done what he always wanted to have or needed.

And this is what the iPKG project is about. We need some easy way to build, package and deploy our projects and iPKG can be a part of that.

This post is about the packaging of software.

The ipkgbuilder project (sister project of iPKG) creates RPM packages which are compatible with the iPKG client (but not with yum).

It is written in Java and needs at least a Java 1.8 Runtime Environment (JRE). It can run either on the PC or on the IBM i server.

The source will be available at the project site on Bitbucket.

There is already a precompiled binary in the download section at the project site.

Connection Properties

The connection.properties file contains the following entries:

  • server
  • user
  • password
  • buildlibrary

If you are running the ipkgbuilder on the IBM i server you can drop the entries server, user and password as it uses the current job for the connection.

Spec File

The spec file is a well formed XML file which contains everything about the content of the RPM file.

The structure of the XML is pretty flat. It has a top element package with a couple of sub elements.

  • name : package name
  • version : package version
  • buildVersion : build version
  • group : category of the package, see list of categories/groups
  • summary : summary of the package content (one liner)
  • description : more detailed description of the content
  • license : license of the software
  • url : URL of the software (like project website)
  • vendor : vendor or author of the software
  • packager : name of the packager (which may be the same as vendor but don't have to)
  • osVersion : Version of the IBM i operating system
  • requirement : dependencies of the package
  • provides : specifies what the package provides besides itself (see virtual packages
  • dataPackage : mark package as a data package
  • directoryPrefix : The directory prefix specified the part of the file path which can be replaced on installation, see relocation.
  • exportedModule : see iPKG documentation about IPKG binding directory
  • exportedServiceProgram : same as exportedModule
  • file : absolute file path of each packaged file or object
  • purgePrefix : remove beginning part of the file path
  • preInstallScript : QCMDEXC script, see iPKG documentation about scripts
  • postInstallScript : QCMDEXC script, see iPKG documentation about scripts
  • preUninstallScript : QCMDEXC script, see iPKG documentation about scripts
  • postUninstallScript : QCMDEXC script, see iPKG documentation about scripts

At my spec files repository you can see the spec files used for the repository at RPG Next Gen.

Example spec file of the stream project:

<?xml version="1.0" ?>
<package>
    <name>stream</name>
    <version>1.0.0</version>
    <buildVersion>1</buildVersion>
    <group>Development/Library</group>
    <summary>Streaming API for ILE</summary>
    <description>A stream consist of an emitter which provides the data for the stream. The data can be piped through pipes and may end in a sink.</description>
    <license>MIT</license>
    <url>https://bitbucket.org/m1hael/stream/</url>
    <vendor>Mihael Schmidt</vendor>
    <packager>Mihael Schmidt</packager>
    <osVersion>7.3</osVersion>
    <requirement>arraylist &gt;= 2.0.0</requirement>
    <requirement>arraylist &lt; 3.0.0</requirement>
    <requirement>message &gt;= 1.1.0</requirement>
    <requirement>message &lt; 2.0.0</requirement>
    <dataPackage>false</dataPackage>
    <directoryPrefix>/QSYS.LIB/MIHAEL.LIB</directoryPrefix>
    <exportedServiceProgram>/QSYS.LIB/MIHAEL.LIB/STREAM.SRVPGM</exportedServiceProgram>
    <file>/QSYS.LIB/MIHAEL.LIB/STREAM.SRVPGM</file>
    <purgePrefix></purgePrefix>
</package>

Package Version

The version is best specified in the semantic versioning scheme, like x.y.z where x is the major version number, y is the minor version number and z is the revision number. The version can also be specified as x.y or just x. Only numbers are allowed in the version number. No other characters and no version qualifiers.

Build Version

The buildVersion is the version of the package used by the packager to identify differences in packages to previous versions of the package. The original content of the package may be the same but the packager may have patched it and thus the package needs to be marked different from the previous package. The build version consist of a single number.

OS Version

The osVersion is also specified using the semantic versioning scheme, f. e. 7.3 .

Requirements

Each requirement is specified in a new requirement element. See the iPKG documentation what can be specified as a requirement and how.

Data Package

A data package is treated as the other packages. The only difference is the handling of the location where the content is placed. The objects will be restored in the data library specified on the command, see parameter DTALIB. Valid values are true and false.

Build

There is a ready to use jar file in the download section at the project site. The program needs the following parameters:

  • connection.properties : Properties files which contains the connection settings and the build library
  • spec file : XML file which contains the definition of the RPM file
  • output directory : Directory for the RPM file

Example:

java -jar ipkgbuilder.jar connection.properties my_project.spec packages_directory

Note: The name of the RPM file is determined by the attributes in the spec file, like name, version, etc... .

QSYS.LIB

Objects from the QSYS.LIB file system are packed into a save file in the build library. The save file is the payload of the RPM package.

IFS

All other stream files are packed into a zip file in the /tmp folder. The zip file is the payload of the RPM package.

Check RPM Entries

You can check the entries of the RPM file with the embedded redline library.

java -cp ipkgbuilder.jar org.redline_rpm.Scanner my_project-1.0.0-ppc-ibmi-7.3.rpm

The Scanner program outputs all RPM header entries (and chokes on the payload).

Repository

One of the reasons for choosing the RPM format was the existing tools available on various platforms to generate and process RPM packages.

One of those tools is createrepo which takes a set of packages and generates some XML files about them. Those XML are processed by the iPKG client to get some information about the repository and the packages hosted in that repository.

Note: createrepo is also available on IBM i via yum.

Wrap it all together

A build script for a repository might look like this:

#!/bin/bash

rm -rf repository/repodata

SPECFILES=$(ls specfiles)

for SPECFILE in $SPECFILES
do
    echo "Building from spec file $SPECFILE ..."
    java -jar ipkgbuilder.jar connection.properties specfiles/$SPECFILE repository/
done

createrepo --no-database --content ibmi --distro rpgnextgen -p ./repository/ -s md5

# ipkg doesn't process gzip files yet => need to unzip
gzip -k -d ./repository/repodata/*.gz

# provide a convenient zip file for offline installation
rm -f rpgnextgen-repository.*
zip -r rpgnextgen-repository.zip repository
md5sum -b rpgnextgen-repository.zip > rpgnextgen-repository.md5

# upload all to hosting platform
ssh rpgnextgen.com@ssh.rpgnextgen.com "rm -rf /www/repo/repository/*"
scp -r repository/* rpgnextgen.com@ssh.rpgnextgen.com:/www/repo/repository/
scp -r repository/repodata/*primary.xml rpgnextgen.com@ssh.rpgnextgen.com:/www/repo/repository/repodata/primary.xml
scp rpgnextgen-repository.* rpgnextgen.com@ssh.rpgnextgen.com:/www/repo/repository/

# purge cache on hosting platform
ssh rpgnextgen.com@ssh.rpgnextgen.com "cache-purge https://repo.rpgnextgen.com"

What's next?

Now we can easily script the build of RPM packages. The next level of package creation automation would be the integration into a build server like Jenkins. Jenkins uses pipelines to define the build script. So having a pipeline command for creating an RPM package would be great.

What do you think? Please send me your feedback!

Happy packaging!

Mihael

Tags : RPG