Integration and the art of ArchiMate

So, you’re a minor integration deity, you’ve got your ArchiMate certification and now you’re prowling the internet in search for whoever was first with depicting integration architecture artifacts into ArchiMate. Chances are, actually, you won’t find anything remotely matching your level of integration knowledge. At first, you even start doubting your google skills, but no, it’s just that ArchiMate isn’t that widely used yet or incorporated into shared models.

As you might have guessed, the above happened to me. Fortunately, I was working for an organization that required the use of ArchiMate when sketching architecture items in their architecture solutions documents. And the specific project I was assigned to, consisted of just that: creating an architecture solutions document for a specific integration context.

So again, I found myself browsing the organization’s ArchiMate repository for any reusable integration architecture context. Now guess what there was to be found on this subject? Right: Nothing.

Now please bear with me. I’ll come to exploring integration in ArchiMate shortly. It’s important that I first clarify that I don’t want to just present an end-state of the universe of integration architecture in ArchiMate, but rather an exploration of my personal integration knowledge together with my evolution of ArchiMate experience.

In order to supply some structure, I am splitting up this evolution into the following three separate parts, of which II and III will get their own blog:

I. Where integration went wrong: ArchiMate and backward integration. You know, the ESB.

II. Some more of integrations greatest mistakes: ArchiMate and integration solution design.

III. What is this integration thing anyway? ArchiMate and a reference architecture for integration.

Part I – Where integration went wrong:
ArchiMate and backward integration. You know, the ESB.

Just because many integration platform users are moving away from the ESB pattern, doesn’t mean the idea itself is backward. It’s mostly the factual more EAI oriented implementation or (lack of) governance of the ESB’s guiding principles that rendered it obsolete in the wake of the federated integration platform implementations we see in the present time. But because I started my ArchiMate exploration with this typical ESB implementation at hand, then that’s what you get for a first integration-into-ArchiMate model: a mix between EAI and service orientation.

Here is my first step of setting up an integration architecture view in ArchiMate. I’ve aggregated a generic provider and consumer (both applications), who collaborate in some way in order to fulfill some E2E application process:

You see, it says it all actually: Basic integration like this is about the collaboration of two or more applications in order to fulfill some application process that in turn fulfills some business function, process or service:

For aesthetic purposes I’ve moved the provider and consumer applications into the collaboration. Unfortunately, this way you’ll not be able to see if it’s an aggregate or composition, but that’s not important right here.

Also, note that the implementation of the end-to-end process is not further elaborated.

Now for the capability that implements this application collaboration, I’ve modeled it as follows:

1.  It’s not the collaboration of applications that is implemented but rather the specific interaction between a consumer and a provider.
   And lo and behold, here the specific integration pattern emerges.
2. As said, an ESB implements the interaction (the integration pattern).
   You might argue that modeling this way violates the service oriented nature of the ESB, and I agree. This is just how I’ve seen ESBs being used again and again, and over again. It should’ve been called Enterprise Integration Bus or something. I’ll leave it like it is for now.
3. This ESB in turn is composed (!not aggregated!) of a core integration platform and a messaging platform; implicitly depicting that this ESB is capable of and intended for synchronous and asynchronous message integration.
   Note that I’ve explicitly used the composition relation between the ESB and the underlying applications, in a clear but futile effort to disallow not-so-integration-savvy(solution)architects use these underlying applications directly and independently from the ESB.
4. At the technology layer each of the two platforms could be of separate nature (software / vendor) but in the end, conceptually they form, again, a collaboration.
   This collaboration construct too helps with disallowing solution architects to make direct use of the messaging or core integration platform and thus breaking the loosely coupled paradigm under which an ESB operates.
   In general, good governance must be in place to have capable integration platform architects guard correct usage of this pattern. You know who I speak of.

I’ve been using this ArchiMate setup for integration for quite some time now and for quite a few different “integration-heavy” solution architectures. Now let’s explore the following example:

When an organization’s E2E processes touch the ESB at some or even multiple points and something goes wrong, the easiest way to find out, it seems, is to contact ESB operational management and ask them nicely if they can find the culprit application that didn’t accept or react to a certain message. In the end, especially when the ESB’s messaging part is also used for event-like messages, a lot will go wrong and the ESB operators find themselves spending more time digging through logfiles looking for exceptions, than spending time for their regular activities. Funny though that by being capable of finding the problem-causing application, the ESB, or rather “ESB operations”, gain a negative connotation because all problems seem to point at the ESB. But that’s a topic for another discussion. For now, it is important to note that a business case has arisen for faster ESB-problem resolvement. But before just buying a quick-logfile-analysis tool or integration message flow monitoring application, you could start with requirements gathering. Now ArchiMate’s motivation aspect comes in handy:

1. The result of the “find Application problems with ESB monitoring faster” requirements process is shown as: ESB Functional Monitoring Requirements (midway bottom right of picture below)
2. These requirements are input and should be met by some specialized application that implements these requirements.
3. Concurrently, this ESB Functional Monitoring application also implements the ArchiMate function of ESB Functional Monitoring. Sadly however, the “chosen” application does not monitor the messaging part of the ESB and sadly too, ArchiMate does not have a way to show what you don’t want or what should but isn’t. Fortunately, the tool I used allows for highlighting with color and text. Though only the text will show up in other views.

As it happened in my example organization, concurrently with the ESB functional monitoring initiative there was a large program going on of which one of the objectives was the implementation of E2E process monitoring at the business level. When I found out, I contacted the lead architect of this program and got myself assigned as solution architect for this E2E process monitoring part with the following ArchiMate view:

As you can see, I skipped the “ESB functional monitoring application does not implement messaging monitoring” in favor of “ESB functional monitoring application does indeed implement ESB functional monitoring” because I considered it inappropriate and too granular for the purpose. But I didn’t actually remove the first statement from the model. Furthermore, you can see that because the ESB, with this company, was at the heart of many (business)processes, it can serve or is part of an E2E process monitoring solution. And yes, I reused the requirements gathered during the “find Application problems with ESB monitoring faster” requirements process. At least I learned something from service oriented architecture.

So there you have it, the first (my first) shared integration ArchiMate model. Use it to your benefit. Change it as required. Comment to improve it. And above all, please share your models to contribute.

All models used are in one Archi tool model. Drop me a line if you would like to receive it.