In the previous post we started connecting all our company’s systems using EAI. So, here we are, all our systems are now connected. (<sarcasm=on>You wish!<sarcasm=off> It would be nice though, but it seems to be a never ending story). Why did we do this, you ask? Oh yes, our users wanted something. Who are our users again? Within the realm of EAI, our users are business users, not customers. Business users use the systems we connected with EAI.
The customers of our company use the company website or they call the company call center. Who picks up the phone? That’s right: our users, the business users. (“Our” means “of us, the IT boys and girls who make EAI happen”)
Okay, we know our users, now what do they want? Let’s see what it is they do all day. A lot of them, and I didn’t make this up, need two screens, in order to copy some data from one system to another. This is what they needed EAI for in the first place. They don’t want to do this anymore, so they start a project. The main requirement of the project is: whatever the user types in system A needs to be copied to system B. This premise leads to a whole new post about how project thinking kills Service Oriented Architecture (and any other type of architecture) and SOA Governance (and any other type of Governance) which I will have to postpone until the end of the bubbles series.
Processes
What I’m after in this post is the bigger picture of what our users are up to. Let’s revisit the about silos post again. In it I explained that a company has a number of pillars and around and between these pillars are multiple processes. Our users use the not-our-IT (a.k.a. one or more of The Systems) to run these processes. I will zoom in on this sentence, because just reading it like that does not do it justice.
- Our users: should be clear, because I started the post with them.
- Not-our-IT: the best of breed systems that support a pillar, like a CRM system. What is our IT, you ask? All the IT that is attempting to remove the silos: EAI, SOA, BPM.
- Processes: things our users do, like ‘hire employee’, ‘approve loan’, or ‘get new customer’.
So our users perform processes. As long as a process stays within the boundary of one system, the user just uses that system and we (the EAI IT lot) are not involved. As soon as the process uses multiple systems, making it a cross-silo process, the user is in trouble. Well, not trouble, but he has to perform stupid, repetitive actions, so he causes trouble.
Example time
Why wasn’t this a problem before now? Ah, that’s why I wrote the about-silos post first, so I could keep on referring back to it: a lot less was automated. Let’s hire someone 40 or 50 years ago. Say hi to Mary. Mary will start working for our company. What does Mary need before she can start her work? What needs to be taken care of? Quick brainstorm: contract, salary, chair and desk, typewriter and paper and she’s good to go. In other words: some paperwork and some goods. Who takes care of this? The office clerk and secretary.
Process: type some stuff, file some stuff, order some stuff.
Now, hire Mary today, if she’s not retired that is. Mary will need mostly the same stuff, but also a lot more. I will add up the systems needed in brackets: access card for the building (1), salary administration (2), account for laptop (3), access to certain systems and files, but not others (4), email account (5), and probably a lot more. The process of hiring Mary used to be a checklist on paper, now it’s some half-automated, half-hidden monster process. Nobody knows for sure if all steps are completed successfully and only IT operations can solve issues by reading the correct log file. Poor Mary.
Automate the process
Clearly, this process needs to be fine-tuned, written down, automated and tracked, which is exactly what BPM is all about. For details, see the BPM crap circle on the right.
The promise of BPM is that all of your business processes are from now on visible for everyone, understandable for everyone and completely automated. Except that this never happened. I hardly come across a working, highly advertised, look-what-we’ve-got, BPM implementation. What went wrong?
Well first you need someone who knows your processes. And I mean really know them. Besides this person you need someone who can model the processes. Ideally this is the same person, but usually it’s not. You see, BPM is written down in BPMN (the notation language of BPM, clearly explained by Bruce Silver) and this notation seems straightforward, but more and more had to be possible, so it really is not.
After modelling comes execution. Boy, when that works… silver bullet. Anyway, what do you need for execution? Connection to all of your systems. Systems that aren’t designed for connection and each speak a different language. The solution is to put our EAI experts to work and redraw the process so that it actually works. And then the redrawn process is no longer understandable for anyone.
Then last, but not least, comes using the BPM process. This is the best part. EAI consultants have automated the process as they understood it. They will now try to show it to the users. After all, the users are the ones who are supposed to work with it. Let’s illustrate both sides here, or else you’ll miss all the fun. The users, the business people, have until now used their custom made screens for 15 years. I can just hear them saying right now: “CRTL+SHIFT+F5 lets me jump straight to this search field. I use that at least 30 times a day.” They see and use screens! They do not use a process! They see their screen all day, every day and they’re used to having their screen that way.
Then there are the EAI experts. I’m talking about real tech-savvy people here, not an ounce of empathy between them. “What! All you talk about is the logo and that it’s not at the top of the screen, but at the bottom? WHO CARES? We spend 3 freaking weeks on the process! Will you please just judge the process and how it works! Forget about the screens.”
The process is not the workflow
I was there when it happened, exactly as I described it. It was real. So what on earth happened there? I call this course: “Viewpoints and how they aren’t the same for everyone”. Users of the system don’t see the system. Users within a process don’t see the process. They see their workflow. Users open their inbox and there it is: all the work they have to do that day. Here it is now: 1) open the issue on top, 2) read the issue, 3) click on the link at the bottom, 4) solve the issue (make a call, type something, search something, whatever), 5) close the issue, 6) drink coffee, and repeat. Workflow. How do the issues get in their inbox? Who knows, who cares? Where do the issues go when they are finished? Who knows, who cares? And then along comes progress for progresses sake and I need to change? No Way José!
Then there is the process, e.g.: 1) customer wants a loan, 2) customer fills out a form at your company’s website, 3) somethings are checked automatically, 4) some things are checked by person X (put in inbox of person X), 5) person X approves -> go on, person X rejects -> end process, 6) give loan to customer.
The other thing about BPM
Besides all this fun, there are also some nasty quirks in most BPM products which are really annoying. E.g. after you model your process, it turns out that the process becomes really slow if you hold too much data in your flow. This means some technical guys need to solve this. Then you need to connect to all these systems, through your middleware (EIA) solution to get the data you need. Again, to be solved by some technical guys.
And then there’s the screen thing. BPM products come with the possibility to use their own build-in screens which you can’t really adapt to the user’s needs, or: you can build your own custom solution. When you take that road, someone will always say something like: “Why don’t we just build the whole process using this technology?”
So the promise was: all of your business processes are from now on visible for everyone, understandable for everyone and completely automated. And what you got was: it doesn’t perform, nobody likes the screens, and after switching to the other technology, nobody can actually see the process anymore.
And then there are the business rules engines. Let’s not go there…
So BPM always sucks?
No, not at all. From experience I know it can be done. How then should we approach it? That’s something for another post.
Just kidding, I’ll give some pointers. Firstly, BPM is not EAI. What? You just said EAI expert were writing down the process?!? Yes, I did, but they really shouldn’t. The processes should be modeled by BPM experts (meaning a business user and a modeler), because BPM processes contain business logic. And business logic should be visible, clearly readable and understandable from the code or configuration, and easily adaptable by our business users. All the technical stuff should be hidden from our business users.
Secondly, a BPM-product expert should be available to point out, at design time, that certain choices made in the process will not perform. Certain orchestration should be moved to the EAI layer. Don’t keep too much data in your process, but limit your data to id’s and keep large data-objects easily available in a cache.
My last tip: use UX experts for screen design, use UX software to build screens and use UX testers to gather screen feedback from the users.
After reading this post you might get the feeling that I don’t like EAI, BPM, or SOA at all. Au contraire. I love this stuff and all the challenges that come with it. It’s bad implementation I don’t like. What I’m trying to accomplish with these posts is to point out that it is possible to implement a BPM solution that has real added value, just that it’s not straightforward. Merely buying the software is not enough. You also need people who know what they are doing and then you need to listen to them.
(Hmm, it looks like I need to start a new set of posts: “using it properly: BPM”. To be started after this series…)
Where does that leave us in the silo game
One of the drawbacks of using BPM on top of existing silos was that it is hard to connect properly to all the systems. During your process you need some business information. It shouldn’t matter what system holds that information. What matters is that you get it and that it is clear from the name of the information what it is. The same language across the company, not some systems-specific name that nobody understands.
So two things are needed: a common model for my company data (CDM) and a system-agnostic manner in which to get the data that I need for my process. (SOA can be used for that)
Not coincidentally, these will be the topics of the next two posts: CDM and SOA.
