Being a good consultant, I always try to avoid repetitive, manual work. What helps me not to just try, but also to succeed, are the many very useful tools out there, on the internet, or provided by colleagues. However, sometimes even the internet cannot give you an answer for your problem. I encountered this when creating Dockerfiles for some TIBCO BWCE projects. This was taking so much time, and even causing errors every now and then, that I couldn’t help but to think: there has to be a better way! Other people surely must have encountered this as well.
As it turns out, they didn’t. Or, they did, but didn’t post the solution online. So in this blog I want to share the route I took to create (large) files in relation to Dockerfiles and TIBCO BusinessWorks Container Edition (BWCE). Since you’re reading this blog. I’ll assume a basic knowledge of BCWE. Therefore I will not go into what containers are or what a Dockerfile is. I’ll only focus on the creation of a Dockerfile (and other related container files).
In the beginning, when testing my first solution, it was quite easy to fill in my first Dockerfile by hand.
For example:
FROM tibco/bwce-ftl:latest
MAINTAINER Tibco
ADD FTL_Subscriber.application_1.0.0.ear /
ENV BW_PROFILE=”Docker”
ENV urlFTL=http://192.168.99.100:13131
ENV endPoint=”EP_subscribeFile_2″
ENV durableName=”storedAllFiles_2″
For the learning experience, it was quite useful to create a Dockerfile by hand, and to understand what needs to be in one. For example, the ENV settings are name-value pairs that will be used when creating the container image. So durableName=”storedAllFiles_2″ means that the global variable “durableName” will be filled with “storedAllFiles_2”, based on the Docker profile from the BWCE project. This Docker profile is related to the BW_PROFILE in the above mentioned Dockerfile.
However, when I started creating my first BWCE project, I realized the amount of global variables was a lot larger than expected. This became quite annoying when creating the Docker profile in the properties part of the application in the project.
(Please note that for the Docker profile the variable names needed to be between #, and not %% as sometimes is described by TIBCO.)
I chose to give the variable useful explanatory names, instead of variable1, variable2 etc. This means that for the Dockerfile, I needed to add all the names from the Docker profile (and stands between the # sign) in the Dockerfile, and with its value behind it.
So, the Dockerfile I had to create contained all the global variables used in the project. As me and my team setup the project in such a way that all that can be configured will be configured, this could result in up to 100 global variables needing to be filled in in the Dockerfile. It would take ages to type them, not counting all the errors that come with manual work. And this was not the only file we needed to create…
As we worked with GitLab, we were also using these variable names and their values to overwrite the default values (set by the Dockerfile) for environment specific variables. And, as we know, a yaml file is quite strict in its syntax so an error is made quite easily when adding new variables to the file.
– export APP_LOGGING_CONNECTIONS_FTL_APPLICATIONNAME=Log
– export APP_LOGGING_CONNECTIONS_FTL_USERNAME=anyone
– export APP_LOGGING_CONNECTIONS_FTL_REALMSERVERURL=http://localhost:8080
This was an extra reason to look for a solution that could provide me with:
- The docker.substvar file
- To be used as the profile within my BWCE project so I don’t have to create this profile myself within the project;
- A Dockerfile with the ‘ENV’ settings for all the name-value pairs;
- A yaml file for the domain/environment specific values (which will override the default settings, set by the Dockerfile).
I did an online search for a generator that could create (one of the) files based on some input. Which would save me hours of typing in the entire set of name-value pairs, as well as prevent me from making the inevitable errors that come with manual labor. I didn’t discover anything that could help me with creating these files easily, or in fact, at all. So I chose to create such a thing myself.
I started with a new project within BWCE, which I called “PropertyGenerator”. As input, it starts with the default.substvar file which is available in the project folder from the original project (<<projectname>>.application\META-INF). In this file, all your global variables are stored based on your sourcecode, so this is the truth for your project.
We need to map this source file, an xml file (default.substvar), to another xml file (Docker.substcar) and two plain text files (the Docker- and the yaml file). BWCE was the perfect tool for this, because in one run, we can at once get the three generated files and make sure that all the name-value pairs are consistent (all using the same variable names).
The result of the PropertyGenerator is to create three new files (based on the default.substvar):
- The docker.substvar which will be used within the project;
- The Dockerfile which will be used for the original docker image;
- The yaml file which will be used for starting up the container, and will house the domain variables.
- Note that this generated yaml file is the complete list of all the global variables from the default.substvar, but the global variables are created according to the yaml syntax, and have the correct same names as created in the Dockerfile. This way it’s easy to copy these yaml lines into the official yaml files when deploying to a specific environment.
To illustrate the files creation, you see a successful run in the image below.
The result are new three files, created from the input file default.substvar:
The Docker.substvar can be placed in the same folder where the default.substvar came from (the META-INF folder). Restart your toolkit and in the application part of your project, under properties, you’ll find the new profile Docker which contains the global variables. Be careful in changing values over there, as it will not be the same anymore as in the other two created files, the Dockerfile and the yaml file. If you change a value within this Docker profile, you’ll need to manually change it in the Dockerfile and yaml file as well.
In this case, the name between the # should not change, as it is also mentioned in the Docker- and yaml file.
Please note that if you reload a shared module which is already in your project, it will reset your Docker profile to the default values (taken from the default.substvar). To overcome this, just generate the docker.substvar again with your tool, or copy the previously generated file to the META-INF folder of your project (and reload your environment).
It is as easy as that, to give you a timesaves as well as the minimization of errors in your configuration files!
This pet project has already shown its value by being a real timesaver for everyone in the team. I hope it will serve you just as well. If you’d like want to know a bit more about how we solved this problem, please don’t hesitate to leave a comment.
