Fabric8 is a cloud platform which enables you to create and manage a cloud based microservices infrastructure. This article explains the concepts behind Fabric8 and how it tries to solve the challenges that arise when working in a “modern” IT infrastructure and organization. And most importantly whether or not Fabric8 actually has the potential to solve these issues.
Development and release cycles are becoming shorter and shorter. Delivering the right business value is what software development is now all about. Where traditionally a software development project was structured in long cycles containing different well defined phases like ‘requirements gathering and analysis’, systems architecture and design’, system development, ‘system test’ and ‘system release’ containing the entire scope of a system. The downside of this approach has been well documented by, among others, the agile movement. But to summarize it, the change of delivering something in such a long release cycle has a big risk of being misaligned with the current business needs.
As a result software development and release cycles have become shorter and shorter. Where it has become more and more important to release a smaller scope of functionality quickly so immediate feedback can be received from the users. The evolution of a system becomes a more gradual approach over time driven by real world feedback of actual production use.
Off course this shift in software development and release has been accompanied with a whole different slough of trends and buzzwords. Some of them purely technology driven, others more from an organizational perspective. In this article we are not going to go into depth in all of these trends and buzzwords, since even naming them would be a difficult task without forgetting one or two. But just to give a small indication of the vast amount of new trends, concepts and buzzwords we are dealing with here:
- agile
- scrum
- devops
- cloud
- lean startup
- containerization
- microservices
- continuous improvement
- design for failure
- scalability
- infrastructure as code
The trends we are going to take a brief look at here are Microservices and Containerization.
One of the defining explanations of Microservices comes from Martin Fowler in his blog post about Microservice architectures. Where he outlines Microservices as:
“In short, the microservice architectural style [1] is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies. ”
Which brings us to Containerization. When looking at some of the characteristics Martin Fowler talks about in Microservices: “each running in its own process ”, “independently deployable”, “may be written in different programming languages” it becomes clear that traditional application runtimes and servers will have difficulties realizing a Microservices architecture. Even with the advent of VM virtualization it becomes quite a challenge since VM’s are still quite large and heavy.
Enter containers, containers offer a lightweight easily reusable software packaging and runtime environment. There are a few different technologies offering containerization, however the elephant, or rather the whale, in the room is Docker. The definition from the Docker website:
“Package your application into a standardized unit for software development Docker containers wrap a piece of software in a complete filesystem that contains everything needed to run the application: code, runtime, system tools, system libraries – anything that can be installed on a server. This guarantees that the software will always run the same, regardless of its environment.”
However running an entire system comprised of dozens or even hundreds of different microservices has its own challenge, even when using Docker or some other container technology. Docker only deals with packaging your mircoservice in a container, running, scaling and up-/downgrading these containers especially running containers on clusters of servers becomes challenging. Thankfully the people at Google have had these difficulties for well over a decade since they made the choice of running all there systems in containers. At Google they developed a container orchestration engine called Borg. A couple of years ago they created a special open source release of there container orchestration engine called Kubernetes (or Helmsman).
So what is Kubernetes , in a nutshell (from the Kubernetes site: http://kubernetes.io/docs/whatisk8s/)
Kubernetes is an open-source platform for automating deployment, scaling, and operations of application containers across clusters of hosts, providing container-centric infrastructure.
With Kubernetes, you are able to quickly and efficiently respond to customer demand:
- Deploy your applications quickly and predictably.
- Scale your applications on the fly.
- Seamlessly roll out new features.
- Optimize use of your hardware by using only the resources you need.
Our goal is to foster an ecosystem of components and tools that relieve the burden of running applications in public and private clouds.
This article does not go into the details of both Docker and Kubernetes, since these technologies are so comprehensive they warrant their own article. However, comprised to say both Docker and Kubernetes focus on the runtime aspect of things. Developing software, packaging it in a Docker container, releasing it in a Kubernetes cluster and monitoring it propertly are not part of the Docker and Kubernetes platforms. Which is understandable, since those things are not what Docker and Kubernetes do. However that does leave a gap to bridge to enable developers to become fully productive and enter the world of fast pace release cycles we have talked about in the beginning of the article.
We need a developer platform!
The developer platform focusses on self service for speed and autonomous teams and to create, develop, test, release and operate microservices. This development platform is fabric8.
When looking at the features of fabric8 we quickly get an overview of the scope and functionality of the platform:
- Create – wizards to create microservices
- Build – packaging into immutable container images
- Release – rolling upgrades across teams and environments
- Runtime – service discovey, scaling, failover, load balancing
- Manage – centralized logs, metrics, alers, tracing, circuit breakers
- Feedback – dashboards and metrics to get feedback!
- Platform – on premise, public, or hybrid cloud
Fabric8 technology stack
Fabric8 is an open source development platform specifically tailored for Microservices. However it is not developed from scratch, and offers a ‘best of breed’ approach to what technologies it uses under the covers.
Not surprisingly by reading this article, the core platform fabric8 leverages is based on Docker and Kubernetes. Next to these the core of fabric8 consists of Jenkins (https://jenkins.io/) which provides CI/CD through the Jenkins pipelines.
We already briefly looked at Docker and Kubernetes, so now we will quickly look at Jenkins, and how fabric8 uses Jenkins to implement a CI/CD release pipeline for microservices in a Kubernetes cluster.
Jenkins is the orchestrator of the build, test, promote and approve steps to rollout your microservices into different environments and eventually off course a production environments. To realize this fabric8 uses the Jenkins 2.0 pipelines. These pipelines offer a Groovy based DSL for defining the various steps that form the entire release pipeline. This also includes the building of the docker containers (or Kubernetes Pods), configuration settings and Kubernetes instructions based on template files. This pipeline translate into a single Jenkins job whenever a build is triggered.
It is possible to use a single Jenkins pipeline to completely define the entire release cycle including different testing, staging and production environments using different stages in the Jenkins pipeline. When using these stages, fabric8 offers a graphical overview to monitor the progress of these pipelines.
Fabric8 not only offers the possibility to create and tweak your Jenkins pipelines completely to your liking using the Groovy DSL, but it also offers ready out of the box Pipeline templates to quickly get you off your feet and running your microservices in production in no time!
These reusable pipeline templates are one of the great features of fabric8. Where you can use the fabric8 web console to select the Jenkins pipeline of your choosing.
With the combination of Docker, Kubernetes and Jenkins fabric8 sure has a solid core framework on which to implement a Microservice based architecture. But even though these three components for the core of fabric8 they are hardly the only ones within fabric8.
Kubernetes, Jenkins and Docker form the core of fabric8
When looking at the other parts of fabric8 they have been grouped in a couple different categories:
- Development
- Spring boot – Spring based microservices
- Spring cloud – Kubernetes integration with Spring cloud
- Function – lambda style event based programming model for Kubernetes
- Wildfly Swarm – JavaEE based microservices
- Kubeflix – Kubernetes integration with Netflix OSS Hysterix, Ribbon, Turbine
- Continuous improvement
- Developer Console – Developer Console to help you create, build, manage microservices with deep visualisation into projects, apps and environments
- Elasticsearch – Elasticsearch is a distributed fault tolerant scalable search engine for logs, events and documents
- Kibana – Lets you view, filter and search all logs and events that have been written to Elasticsearch
- Chaos Monkey – Randomly kills pods to help check your environment can withstand failures
- Prometheus – Service monitoring, metrics and alerting system with a time series database
- Graphana – Rich metrics dashboard and graph editor for Prometheus and InfluxDB
- Social
- Slack – Public hosted chat service
- Let’s Chat – Open source on premise chat service
- Systems availability
- Docker – Linux container API, runtime, tooling and image registry
- OpenShift – A PaaS (Platform As A Service) based on Docker and Kubernetes
- Kubernetes – Container orchestration
- ManageIQ – Cloud management
- Automation
- Jenkins – Continuous Integration and Continuous Delivery using reusable Jenkins Pipelines and Functions
- Hubot – Chat bot with gateways to various chat services like Slack, Let’s Chat, IRC and many others
- JBoss Forge – A Java toolbox to help create apps faster
- Nexus – Maven repository manager for Canary and Promoted releases along with a mirror of central maven repositories
- Arquillian – System testing of Docker containers and Kubernetes resources in JUnit and Jenkins
- Collaboration
- Gerrit – Code review of changes
- Gogs – On premise git repository hosting
- Taiga – Project Management Tool (i.e. issue tracker and kanban board)
Going over all these items individually would explode this article, the fabric8.io website provides a good reference on what these various components do and how they are used inside fabric8. However, one of the more stand out features of fabric8 is definitely the logging and monitoring part of the stack. One of the complexities of a microservice based architecture of any distributed architecture for that matter is logging and monitoring. Where having a monolithic system you only have to worry about the health of one system and analyse the logs of one system this complexity exponentially increases in a distributed environment. Where not only you would have dozens or even hundreds of individual components, a lot of these components have loose relationships with other components. Keeping an eye on the performance and health of a distributed system is a big challenge.
Fabric8 has leveraged several open source projects to implement a ‘best of breed’ monitoring and logging solution for all microservices running inside the fabric8 cluster.
Like mentioned above this solution is comprised of the following components:
When the “monitoring” component (read: microservice) is installed in fabric8 all logging of all your containers running your microservices is stored in an Elasticsearch cluster and Kibana is used for creating various graphs and dashboards used for analyzing your microservices:
Regarding metrics, in fabric8 these are collected using the Prometheus project, and dashboarding is provided by Graphana:
In his blogpost Dirk Janssen goes into more detail how to setup and use the logging and monitoring inside fabric8 and use to keep a close look on your microservices and the cluster as a whole.
Technologies on fabric8
Since fabric8 uses Docker and Kubernetes under the covers, basically everything that runs inside a Docker container, which is pretty much everything, runs in fabric8. However, there are some quickstarts and Jenkins pipelines out of the box for certain technologies to quickly get you started developing and releasing microservices. Currently these include:
The Quickstarts have even more out of the box templates available:
With these templates and quickstarts you can create your first microservices and deploy them with an out of the box deployment pipeline with a wizard in just minutes!
Microservices for your microservices
Taking on a “eat your own dog food approach” above we took a brief look at the core components of fabric8, it is worth to note that all these different features of fabric8 used for better and faster microservices implementations also run as microservices themselves inside the platform. This makes the fabric8 platform highly modular and scalable. Users are able to run the platform microservices they need for their needs therefore customizing the fabric8 platform to their liking. Installing these microservices is made very easy via the fabric8 console. A graphical page displays all applications where you can select them and run (install) them in a particular team.
These microservices then run as a Docker container in a Kubernetes Pod inside fabric8, enabling all the features of Kubernetes and fabric8 for these platform microservices as well!
When starting out with a clean install the base framework runs inside pods as well, this contains of fabric8 itself, JBoss Forge, a docker registry, and the CI/CD pipeline containing of Gogs, Nexus and Jenkins.
Some final thoughts
Using a ‘best of breed’ approach to deliver a full microservices stack fabric8 enables IT teams to deliver business functionality at a fast pace. Therefore ever shortening the length of the release cycle. It also helps focus teams on the actual business functionality and so unburden them from all non-core, but still crucially important aspects of software development and maintenance, like monitoring an scaling.
With a core comprised of Docker, Kubernetes and Jenkins fabric8 uses widely adapted technologies backed by large (enterprise) communities. This makes fabric8 a future proof and robust platform for delivering software in the 21st century.
