Open Source on IBM i

Thread Local Storage and ILEastic

Thread Local Storage (TLS) is a widely used programming method that uses static  or global memory local to a thread. — Thread Local Storage on Wikipedia.

So TLS is all about having a storage area per thread. This is not much of a concern for most RPG developers. Most of us are working with jobs and not directly with threads. But if you take a look at the details of a job (DSPJOB OPTION(*THREAD)) you will see that your job has one thread, the main thread. And all your programs are executed in this main thread.

So what good is TLS for you? It can help you even if you have only your main thread. You can put something in that storage and at a later time you can retrieve it. This is really helpful if you have something you need to save at some point in your programs (f. e. at login) and don't need it 99% of the time. But you don't want to pass this data around as a parameter to all programs or procedures. So putting that in the TLS and retrieving it 10 entries deeper in your callstack can make things easier.

But many readers will now say "Hey, we already got that covered. We got LDA and GDA for this.". Yes, you can use LDA for this and it is a nice for storing data at the scope of the job. But how much data can fit into the LDA. Not very much. And how often did you step onto the toes of another application storing data in the same LDA position, f. e. 3rd party software?

Implementation

There are many ways how to implement TLS. A very easy implementation can be done in RPG. Yes ... your eyes don't betray you. It is good old RPG. And we even need only a single control option for this:

thread(*concurrent)

This will make your RPG module thread safe. From the IBM Knowledge Center:

If THREAD(*CONCURRENT) is specified, then multiple threads can run in the module at the same time. By default, all the static storage in the module will be in thread-local storage, meaning that each thread will have its own copy of the static variables in the module, including compiler-internal variables. This allows multiple threads to run the procedures within the module at the same time and be completely independent of each other.

You don't have to do any extra. Just put that control option into your module. Then declare a global variable and more or less you are already done. You can write some procedures to manage access to this variable. But that's it!

"But how is this better than using LDA?"

For example you can use this in a multi-threaded and in a single-threaded environment. It doesn't matter. LDA will not work in a multi-threaded environment or should I rather say it will have some side effects as each thread accesses the same data area.

And as you implement it yourself it is totally up to you what features your TLS has. F. e. you can use a noxDB graph as your TLS. This means you can even store hierarchical data. Or have one graph branch for each part of your application, f. e. /user for all your user data in the job. So you would query /user/name for the name of the user and /user/roles to get an array of roles the user has or /user/email for his email address. No more toe stepping. It is totally up to the implementation.

You can find a simple implementation here on Bitbucket.org.

"But doesn't a new service program instance gets created with each new activation group?"

Yes. But if you specify a named activation group for this service program it will always be executed in the same named activation group and everything should be fine.

TLS and ILEastic

ILEastic is a microservice framework for creating web services in ILE languages, f. e. in RPG. Yes, your reading is just fine. No PASE or Java involved. Just some great ILE modules bound together in a nice service program. ILEastic creates a new thread for each request (as most web and application servers do today).

So this is where our thread local storage comes into play.

You can do some initialization of your TLS in a plugin which is registered on PREREQUEST.

il_addPlugin(config : %paddr('web_util_tlsPlugin') : IL_PREREQUEST);
dcl-proc web_util_tlsPlugin export;
  dcl-pi *n ind;
    request  likeds(IL_REQUEST);
    response likeds(IL_RESPONSE);
  end-pi;

  dcl-s threadLocal pointer;
  dcl-s username varchar(100);
  dcl-s tls pointer;

  threadLocal = il_getThreadMem(request);

  // The ILEastic JWT plugin places the payload of the JWT under /ileastic/jwt/payload.
  // We don't have to worry about not having a value here because without a proper 
  // token or payload the request is returned with a 401 Unauthorized response and 
  // never reaches this line.
  username = jx_getStr(threadLocal : '/ileastic/jwt/payload/username' : 'unknown');

  tls = tls_getStorage();
  jx_setStr(tls : '/user/name' : username);

  return *on;
end-proc;

Means it will get executed before the request is handled by our end point / route procedure. If you need to do some cleanup you can register a plugin on POSTRESPONSE.

il_addPlugin(config : %paddr('web_util_tlsFree') : IL_POSTRESPONSE);
dcl-proc web_util_tlsFree export;
  dcl-pi *n ind;
    request  likeds(IL_REQUEST);
    response likeds(IL_RESPONSE);
  end-pi;

  tls_freeStorage();

  return *on;
end-proc;

ILEastic has its own TLS. You can access it by calling il_getThreadMem(). You get a noxDB graph which you can use as you wish. Only the branch /ileastic is reserved for ILEastic itself.

But you don't want to have ILEastic as a dependency for every program or service program. Especially not those which have nothing to do with web services. So having ones own TLS is a much cleaner solution.

Note: When registering plugins be aware that they will be called by ILEastic in the same order you are registering them.

Use Case

A good use case for this would be the following:

  • You have created a web service with ILEastic.
  • This web service uses a JWT token for authentication.
  • ILEastic checks the token and puts the token data into its thread local storage by using the ILEastic JWT plugin.
  • You use your own TLS plugin to store that token data (like user id, name, ...) in your own TLS.
  • Down the callstack you access your TLS to store in the database which user has created or changed that database entry.

For example you can use this in a multi-threaded and in a single-threaded environment.

As I have said you can use the same TLS service program in your 5250 interactive and batch programs. You just need to initialize your TLS with the data from the current job. In our use case this would be the user name from the current job. So if you now query /user/name you still get a valid value, regardless of the environment.

Happy threading!

Mihael

Tags : RPG