Unikernels, Software Containers and Serverless Architecture: Road to Modularity

This blog post is discussing the implications of Unikernels, Software Containers and Serverless Architecture on Modularity of complex software systems in a service mesh as illustrated below. Modular software systems claim to be more maintainable, secure and future proven compared to software monoliths.

StackedModules

Software containers or the alternative MicroVMs have been proven as very successful for realizing extremely scalable cloud services. Examples can be found in the areas of serverless computing and Big Data / NoSQL solutions in form of serveless databases (which are often not realized using containers). This has gone so far that upon a web request of a user the software container is started that executes a business function developed by a software engineer, an answer to the user is provided and then software container is stopped. Thus, large cost savings received in a cloud world where infrastructure and services are payed by actual consumption.

However, we will see in this post that there is still room for optimization (cost/performance) to modularize the application, which is usually based still on large monolith, such as the Java Virtual Machine with all standard libraries or the Python environment with many libraries that are in most cases not used at all to execute a single business function.  Furthermore, the operating system layer of the container is also not optimized towards the execution of a single business function as they contain much more operating system functionality than needed (e.g. drivers, file systems, network protocols, system tools etc.). Thus Unikernels are an attractive alternative to introduce cost savings in the cloud infrastructure.

Finally, we will discuss grouping of functions, ie where it make sense to combine a set of function of your application composed of single functions/microservices to one unit. Briefly we will address composable infrastructure.

Background: Software Containers and Orchestrators

The example above of course is a simplistic example and much more happens behind the scene. For example, the business function may need to fetch data from a datastore. It may need to communicate with other business functions to return an answer. This requires that these business functions, communication infrastructure and datastores need to work together, ie they need to be orchestrated. Potentially additional hardware (e.g. GPUs) need to be taken into account that is not all the time available due to cost.

This may imply, for example, that these elements run together in the same virtual network or should run on the same servers or servers close to each other for optimal response times. Furthermore, in case of failures they need to be rerouted to working instances of the business function, the communication infrastructure or the data store.

Here orchestrators for containers come into play, for example Kubernetes (K8S) or Apache Mesos. In reality, much more need to be provided, e.g. distributed configuration services, such as Etcd or Apache Zookeeper, so that every component always finds its configuration without relying on complicated deployment of local configuration files.

Docker has been a popular concept for software containers, but it was neither the first one nor was it based on new technologies. In fact, the underlying module (cgroups) of the Linux kernel has been introduced years before Docker emerged.

This concept has been extended by so-called MicroVM technologies, such as Firecracker, based on UniKernels to provide only the OS functionality needed. This increases reliability, security and efficiency significantly. Those VMs can startup much faster, e.g. within milliseconds, compared to Docker containers and thus are more suitable even for simple use cases of web service requests described above.

About UniKernels

UniKernels (also known as library operating systems) are core concept of modern container technologies, such as Firecracker, and popular for providing cloud services. They contain only the minimum set of operating system functionality necessary to run a business function. This makes them more secure, reliable and efficient with significant better reaction times. Nevertheless, they are still flexible to be able to run a wide range functionality. They contain thus a minimal needed kernel and a minimal needed set of drivers. UniKernels have been proposed for various domains and despite some successes to run them productively they are at the moment still a niche. Examples are:

  • ClickOs: Dynamically create new network devices/functions (switching, routing etc.) within milliseconds on a device potentially based on a software-defined network infrastructure
  • Runtime.js: A minimal kernel for running Javascript functions
  • L4 family of microkernels
  • Unik – compile application for using in UniKernels in Cloud environments
  • Drawbridge – a Windows-based UniKernel
  • IncludeOS – A lightweight Linux OS for containers/MicroVMs
  • Container Linux (formerly: CoreOS):  A lightweight OS to run containers, such as Docker, but more recently based on rkt. While this approach is very light-weight, it still requires that the rkt containers that are designed by developers are light-weight, too. Especially care must be taken that different containers do not only include the libraries necessary, but also only the parts of the libraries necessary and only one version of them.
  • OSv – run unmodified Linux application on a UniKernel
  • MirageOSOcaml based

Serveless Computing

Serverless computing is based on MicroVMs and Unikernels. Compared to the traditional containerization approaches this reduces significantly the resource usage and maintenance cost. On top, they provide a minimum set of libraries and engines (e.g. Java, Python etc.) to run a business function with ideally the minimum needed set of functionality (software/hardware). Examples for for implementations of serverless computing are OpenFass , Kubeless or OpenWhisk. Furthermore, all popular cloud services offer serverless computing, such as AWS Lambda, Azure Functions or Google Cloud Functions.

The advantage of serverless computing is that one ideally does not have to design and manage a complex stack of operating systems, libraries etc., but simply specifies a business function to be executed. This reduces significantly the operating costs for the business function as server maintenance, operating system maintenance, library maintenance are taken over by the serverless computing platform. Furthermore, the developer may specify required underlying platform versions and libraries. While those are usually offered by the service provider out of the box, they need to be created manually by them or the developer of the business function.

Those libraries that provide the foundation for a business function should ideally be modularizable. For example, for a given business function one does not need all the functionality of a Java Virtual Machine (JVM) including standard libraries. However, only recently Java has introduced a possibility to modularize the JVM using the Jigsaw extension that came with JDK9. This is already an improvement for more efficiency when using serverless computing, but the resulting modules are still comparably coarse grained. For example, it is at the moment not possible to provide to the Java compiler a given business function in Java and it strips out of the existing standard libraries and third party libraries only the functionality needed. This still highly depends on the developer and also there are some limits. For other libraries/engines, such as Python, the situation is worse.

The popular standard library (glibc) is also a big monolith library that is used by Java, Python, native applications and kernels that has a lot of functionality that is not used by a single business function or even application. Here alternatives exists, such as Musl.

This means that currently perfect modularization cannot be achieved in serverless computing due to the lack of support by underlying libraries, but it is improving continuously.

Service Mesh

A service mesh is a popular mean for communication to and between functions in serverless computing. Examples for service mesh technologies are Istio, Linkerd or Consul Connect. Mostly this refers to direct synchronous communication, because asynchronous communication, which is an important pattern for calling remote functions that take a long time to complete, such as certain machine learning training and prediction tasks, is not supported directly.

However, you can deploy any messaging solution, such as ZeroMQ or RabbitMQ, to realize asynchronous communication.

The main point here is that service meshes and messaging solutions can benefit a lot from modularization. In fact, the aforementioned Clickos is used in network devices to spawn up rapidly any network function as a container that you may need from the network device, such as routing, firewall or proxying. Those concepts can be transferred to services meshes and messaging solution to deliver only the minimal functionality needed for a secure communication between serverless computing function.

The modularization of the user interface

One issue with user interfaces is that they basically provide a sensible composition of business functions that can be triggered by them. This means they support a more or less complex business process that is executed by one or more humans. In order to be usable they should present a common view on the offered functionality to the human users. New technologies, such as Angular Ivy, supports extracting from a UI library only the needed functionality reducing code size, security and reliability of the UI.

The aforementioned definition of UI means that there is at least one monolith that combines all the user interfaces related to the business functions in a single view for a given group of users. Since decades there are technologies out there that can do this, such as:

  • Portals: Portlets. More structured UI aggregation already at server side
  • Mashups: Loosely coupling of UIs using various “Web 2.0” technologies, such as REST, Websockets, JSON, Javascript, and integrating content from many different services

One disadvantage with those technologies is that a developer needs to combine different business functions into a single UI. However, the user may not need all the functionality of the UI and it cannot be expected that a developer manually combines all UIs of business functions for different user groups.

It would be more interesting that UIs are combined dynamically given the user context (e.g. desk clerk vs stewardess) using artificial intelligent technologies. However those approaches exist in academia since many years, but have not yet been managed to use in a production environment at large scale.

Finally, one need to think about distributed security technologies, such as OpenID:Connect, to provide proper authentication and authorization to access those UI combinations.

Bringing it all together: Cloud Business Functions and Orchestration

With the emergence of serverless computing, Microservices and container-based technologies we have seen the trends towards more modularization of software. The key benefits are higher flexibility, higher security and simpler maintenance.

One issue related to this is how to include only the minimal set of software to run a given business function. We have seen that it is not so easy and currently one still has to include large monolith libraries, such as Glibc, Python  or Java, to run a single business function. This increases the risk of security issues and requires still big upgrades (e.g. moving to another major version of an underlying library). Additionally, also the underlying operating system layer is far form being highly modularizable. Some operating systems exist, but they remain mostly in the domain of highly specialized devices.

Another open question is how to deal with the feature interaction problem as the possible number of combinations and modules may have unforeseen side-effects. On the other hand, one may argue that higher modularization and isolation will make this less of a problem. However, those aspects still have to be studied.

Finally, let us assume several business functions need to interact with each other (Combined Business Functions – CBF). One could argue that they could share the same set of modules and versions that they need. This would reduce complexity, but this is not always easy in serverless computing, where it is quite common that a set of functions is developed by different organisations. Hence, they may have different versions of a a shared module. This might be not so problematic if even in different versions the underlying function has not changed. However, if it changes then it can lead to subtle errors if two business functions in serverless computing need to communicate. Here it would be desirable to capture those changes semantically, e.g. using some logic language, to automatically find issues or potentially resolve them in the service mesh / messaging bus layer. One may think in this context as well that if business functions run on the same node they could share potentially modules to reduce the memory footprint and potentially CPU resources.

Answers to those issues will also make it easier  to upgrade serverless computing functions to the newest version offering the latest fixes.

In the future, I expect:

  • CBF Analyzer that automatically derive and extract the minimum set of VM, uni kernel, driver and library/engine functionality needed to run a business function or a collection of loosely coupled business functions
  • Extended Analysis on colocating CBFs that have the optimal minimum set of joint underlying dependencies (e.g. kernel, driver etc.)
  • Dynamically during runtime of a function making shared underlying modules in native libraries and operating system code available to reduce resource utilization
  • Composable infrastructure and software-defined infrastructure will not only modularize the underlying software infrastructure, but the hardware itself. For instance, if only a special function of a CPU is needed then other part of the CPUs can be used by other functions (e.g. similar to Hyper-Threading). Another example is the availability and sharing of GPUs by plugging them anywhere into the data center.

 

Modularizing your Business and Software Component Design

In this blog, I will talk about modularizing your enterprise from a business and software perspective. We start from the business perspective, where I provide some background how today’s businesses are modularized. Afterwards, we will investigate how we can support the modularized business with software components and how they can be designed. Finally, we will see some software tools enabling component-based design for a modularized business, such as the service component architecture (SCA) or OSGi.

Business perspective

You will find a lot of different definitions of what and how a business can be modularized. Most commonly, business modules are known as business functions, such as controlling, finance, marketing, sales or production. Of course you can view this also on a more fine granular level. Furthermore, we may have several instances of the same module. This is illustrated in the following figure. On the left-hand side the business modules of a single enterprise are shown. On the right-hand side you see the business modules of decentralized organizations. There, the enterprise is split up in several enterprises, one for each region. Business modules are replicated across regions, but adapted to local needs.

businessarchitecture

A module has usually clear interfaces to other modules. For instance, in earlier times you used paper forms to order something from the production department.

One of the most interesting questions is how one should design business modules. Well there is no easy answer to this, but one goal is to reduce complexity between modules. This means there should not be many dependencies between modules, if any. There can be a lot of dependencies within one module. For instance, people work very closely together in the production department, because they share common knowledge and resources, such as machines or financial ones.

On the other side, production and sales have some very different business processes. Obviously, they are still dependent, but this should be done through a clear interface between them. For example, there can be regular feedback from a sales person to the production engineer on what the customer needs.

Clearly, it depends on the economic environment how you define business modules and the organization. However, this environment changes and this means business modules can be retired, new interfaces or completely new business modules be created.

Unfortunately, this is usually not very well documented and communicated in many businesses. Particularly, the conditions why a business has been designed out of a given set of modules and dependencies exists usually only in the head of some people. Additionally, the interfaces between business modules and their purpose are often not obvious. This can mean significant loss of competitive advantages.

Linking Business and IT Perspective: Enterprise Architecture

Business and IT have not necessarily the same goals. This means they need to be aligned, so that they are not conflicting. Hence, you need to map your business modules to your IT components. This process is called Enterprise Architecture Management. During this process the Enterprise Architecture is constantly modified and adapted to the economic environment.

Of course, you cannot map all your business and your whole IT, because this would be too costly. Nevertheless, you need to choose the important parts that you want to map. Additionally, you will need to define governance processes and structures related to this, but this is not part of this blog entry.

One popular, but simple, illustration is an enterprise architecture composed of four architectures:

  • The Business Architecture describes the business functions/modules, their relations within business processes, people, the underlying strategy, business goals and the relevant economic environment.
  • The Information Architecture is about the business data, their relationships to business functions/modules and processes, the people, its value as well as security concerns.
  • The Software Architecture depicts different kind of components according to IT goals, their relations to business data and business functions/modules.
  • The Technology Architecture is about the technology foundation for enabling the other architectures. It describes the basic infrastructure in form of hardware and software components. This includes local environments as well as cloud environments, such as OpenStack, Google Compute or Amazon EC2.

Some people advocate additionally an IT security architecture. I propose to model it not as an additional architecture, but include IT security concerns in each of the aforementioned architectures. This increases the awareness for IT security in your business. Nevertheless, appropriate tools can generate from the models a complete security view over all architectures.

There are many popular tools, such as the ARIS toolset to map your enterprise architecture.

Of course, you cannot only define top-down from business to IT how this architecture should be designed. You need to take into account the IT perspective.

IT perspective

As mentioned, IT and business goals are not necessarily the same. IT focuses on three areas: storing of information (storage), processing of information (computation) and transporting information (network). The goal is to do this in an efficient manner: Only the minimum of information should be stored, processing information should be as fast as possible and transporting information should only consume minimal resources. Clearly, there are constraints ranging from physical laws over business goals to IT Security concerns. For instance, the three key IT Security goals, namely confidentiality, integrity and availability often have negative impact on all three IT goals.

As I have explained before: business modules are mapped to software components and vice versa. One big question here is about the design of software components, i.e. what software functionality (representing business functionality) should be part of one software component and not one of the others. Software components are usually different from the business modules, because they have different design criteria.

In the past, people often used heuristics. E.g. they introduce “data components” and “functional components”. This makes sense, because you should not have 50 different databases, but only the right amount of databases for your purpose (e.g. one for NoSQL, one for SQL and/or probabilistic databases). This reduces the resource needs and avoids inconsistent data. However, there is no general approach how these heuristics should be applied by different enterprise architects. Another problem is that communication patterns (e.g. via message brokers, such as RabbitMQ) are completely left out.

Hence, I think a more scientific or general approach should be taken towards the design of components, because these heuristics do not give you good guidelines for a sustainable architecture. Based on the three IT focus areas, I propose to have software components for storage (e.g. database), computation (e.g. business logic) and network (e.g. message brokers). Once you have identified these components, you need to decide which functionality you should put in which component. One key goal should be to reduce the dependencies between components. The more communication you have, the more dependencies you have between the different functions in components. Evaluating this manually can be costly and error prone. Luckily, some approaches do this for your and they can be integrated with business modeling as well as software component management tools (cf. here an approach that derives the design of software components (managed using the service component architecture (SCA)) from the communication pattern in business processed (modeled using the business process modeling notation (BPMN)).

Another mean for coherent software component design is to have enterprise architects responsible for mapping one part of the business (e.g. controlling) reviewing the software architecture for another part of the business (e.g. marketing). I introduced such a process in an enterprise some time ago. Such an approach ensures that architecture decisions are made consistent across the enterprise architecture and this fosters also learning from other areas.

Finally, a key problem that you need to consider is the lifecycle management of a software component. Similar to the lifecycle of business modules, software components are designed, implemented, deployed and eventually retired. You need tools to appropriately manage software components.

Tools for Managing Software Components

Previously, I elaborated on the requirements for managing software components:

  • Handle interfaces with other components

  • Support the lifecycle of software components

Two known information technologies for managing software components are OSGi and the Service Component Architecture (SCA).

OSGi

OSGi is a framework for managing software components and their dependencies/interfaces to other components. It is developed by the OSGi alliance. It origins from the Java world and is mostly suitable for Java components, although it has limited support for other non-Java platforms. It considers the lifecycle of components by ensuring that all needed components are started when a component is started and by being able to stop components during runtime. Furthermore, other components and their interfaces can be discovered at runtime. However, there is no deployment method for software components part of the standard.

Since Java can run on many different devices, it is available for Android, iOS, embedded devices, personal computers and servers.

Unfortunately, tool support for linking OSGi and business or information architecture is very limited. Furthermore, an automatic generation and deployment of OSGi components from your enterprise architecture does not exist at the moment. This makes it difficult to understand software component and their relations within your enterprise architecture.

Many popular software projects are based on OSGi, such as the Eclipse project.

Service Component Architecture (SCA)

The service component architecture is a specification for describing software components, their interfaces and their dependencies. It is developed by members of the Organization for the Advancement of Structured Information Standards (OASIS). It does not depend on a specific programming platform, e.g. it supports Java and C++. It supports policies that govern components, a set of components or their communication. However, SCA does not consider the software component lifecycle or how they are deployed exactly.

It is supported by many middleware frameworks, such as TIBCO Active Matrix or Oracle Fusion Middleware.

Similarly to OSGi there is little tool support for linking SCA components and the business or information architecture. However, the SCA specification has a graphical modeling guideline and some recent work describes how they can be linked via business processes. Since OASIS is responsible for other enterprise architecture relevant modeling notations (e.g. BPMN), it can be expected that enterprise architecture tools can be adapted to provide support for linking different parts of the enterprise architecture.

Conclusion

Modularizing your business and designing software component is a difficult task. Not many people understand the whole chain from business to software components. While enterprise architecture and modeling has become a popular topic in research and practice, the whole tool chain from business to software components is not. There have been attempts to introduce model-driven-architecture (MDA), but the supported models were mostly only restricted to the Unified-Modeling-Language, which is not very suitable for business modeling and can be very complex. Additionally it does not take into account the software component lifecycle. Furthermore, the roles of the different stakeholders (e.g. business and IT) using these tools are unclear.

Nevertheless, new approaches based on the business process modeling notation and frameworks for managing software components make me confident that this will change in the near future. Growing IT complexity in terms of communication and virtualization infrastructure will require software support for managing software components. Companies should embrace these new tools to gain competitive advantages in terms of agility and flexibility.