If you’ve done your research, you probably know that Kubernetes is only one piece of the puzzle. Production grade deployments require a lot of moving pieces including logging and monitoring, governance, and more. You’ll also need some key extensions — some Kubernetes can’t go without, others that will make your life a lot easier. Let’s take a closer look.
Overlay network plugins
Overlay networks fall into the “can’t go without” category — after all, your containers must communicate somehow.
In most Kubernetes clusters, overlay networks will assign pods IP addresses from a virtual IP address pool. While Kubernetes is able to work with a physical network, assigning pods IP addresses from a physical address space, this still requires some sort of integration implemented through another network plugin. While your cluster can live without an overlay network, you’ll only get the full value of flexible container management if you use one.
Available plugin examples include Calico, Weave, Flannel, Canal (Calico + Flannel), Kube Router, among others. All of which can be installed manually and separately. However, since the overlay network is such an essential part of the Kubernetes cluster — without it, the cluster won’t be of much use for most apps — it should be part of your automated Kubernetes management platform. If not, the burden is on your ops team to set it up separately for each cluster they deploy. While that is possible, it isn’t scalable. So make sure it’s automated and part of the Kubernetes layer.
Cloud native storage
While you may start your Kubernetes journey with stateless apps, you likely won’t stop there. When you venture into stateful apps, you’ll need a way to manage cloud native storage. This is wherecloud native storage extensions come in handy.
In case you are asking yourself about Kubernetes’ native storage capabilities. Yes, Kubernetes does provide a rich set of basic capabilities to work and integrate with different types of external storage. But relying on these capabilities alone will leave a lot of unanswered questions, including storage provision, management of access, or that of SLAs for different storage types.
While these questions can all be addressed in a semi-manual manner, it is an additional burden for your ops team and thus not scalable. Scalable Kubernetes clusters with storage-based apps will require some tool that automates cloud native storage management, operations, and governance. There are several available frameworks on the market. Portworx is the most well-known commercial option, there is also StorageOS, and Robin. Ceph and Rook are popular open source projects but will require some work to deploy, integrate, and operate successfully.
For cloud native storage, you have three options: (1) build your own; (2) leverage a commercial product like Portworx with governance capabilities and easy Kubernetes integration; or (3) use cloud native storage that comes with your Kubernetes platform (although there aren’t many examples).
Since a “build your own” approach isn’t generally feasible — it requires more work and resources than most organizations are able or willing to invest — we recommend an existing cloud native storage framework either from a separate vendor or the one built into your container management platform.
CI/CD pipeline
Next, we move to the application lifecycle management layer with continuous integration and continuous delivery (CI/CD). If you moved to Kubernetes, you likely already implemented CI.
There are a wide variety of CI/CD alternatives. While some tools are cloud native specific, others are of more general purpose and can work with Kubernetes or other deployment tools.
These tools also have different degrees of customizability. Some come with a set of pre-configured pipelines which make it difficult or even impossible to customize. Others are much less restrictive but require more effort setting up.
Before making any decision, look at what the dev team is currently using and comfortable with. You may be able to integrate it with Kubernetes and the cloud native stack in general. If that isn’t the case, we recommend well-known open source tools such as Spinnaker, Jenkins or a combination of both.
Security management and governance frameworks
If you’re an enterprise, you can’t go without security management and governance. Unfortunately, governance is still a pretty patchy area. Different governance rules are implemented through different frameworks. And because it is so scattered, different management and governance extensions may overlap with other security frameworks you already have in place.
With no single comprehensive tool that addresses all your governance needs, you’ll need to mix and match until all critical areas are covered. Your goal is to minimize the number of governance frameworks needed while maximizing the coverage so it’s easier to manage for your ops team.
The same is true with security plugins, although not as patchy as with governance. Different organizations have different security requirements. The first step is to define your requirements and identify the appropriate extensions. Here again, minimize the needed frameworks while maximizing coverage. The more plugins ops has to manage, the more complex it gets.
You have extensions focusing on the apps running in your Kubernetes cluster. Integrated with your CI/CD pipeline’s image management tool, some extensions allow you to define different governance policies, such as which images are permitted for production. Some CI/CD pipeline steps or stages will require security approvals for images to pass to the next stage, which can also be controlled via security extensions.
Certain security frameworks will integrate with those of your overlay network or at least leverage some of Kubernetes’ security capabilities such as network policies, pod security policies, etc. Solutions like NeuVector help automate more generic policies across your clusters, environments and applications.
Ingress management
To make your Kubernetes cluster services available to external users, you’ll need something to manage ingress. In the simplest case, you can leverage an ingress controller deployed within the cluster. In more complex scenarios, you may need multiple ingress controllers and an integration with an API management system (internal and external). NGINX or Kong are great tools for the latter. Both provide integration with Kubernetes, different API management systems and other cloud native tools.
Application runtime frameworks
Then, of course, there are numerous application runtime extensions. Service meshes and serverless frameworks are often used with Kubernetes. Since they are frequently part of the cluster configuration, the ability to deploy those frameworks automatically with each cluster will be beneficial. This is particularly true for dev and QA.
When using service meshes and serverless, keep in mind that they collect additional metric dimensions from your apps and should, therefore, tie into your Kubernetes log collection and monitoring. Service meshes, for instance, provide valuable tracing information by associating an ID to every request and tracing it throughout microservices and serverless calls. That tracing info is readily available and, by monitoring it, you can troubleshoot a wide category of problems that may arise in real-life scenarios.
Image registry is another area. You’ll need it for apps to recover and Kubernetes to restart pods. Not only is this a deployment dependency, but it’s also a runtime dependency. While in a non-containerized world we can live with package repositories only touched when ops sets up or upgrades systems; with the cloud native stack, image repositories are a crucial part of the actual runtime.
Image management, or more broadly artifact management, should be part of your Kubernetes extension essentials list. It goes way beyond just hosting those artifacts binaries, helm packages or docker images. Additionally, you’ll need to consider governance rules around these artifacts. For instance, who can publish in which repositories? These governance rules must integrate with your security framework which is ultimately responsible for governance at large.
Conclusion
Kubernetes extensions or plugins are a key part of your Kubernetes stack. Selecting an overlay network, cloud native storage, or CI/CD framework is more or less straightforward. One tool will do the trick. For security and governance, things are more complex. There is no one comprehensive tool that solves security or governance as a whole. You really have to think it through, identify all security requirements and patch it together. Given how important that is, this task shouldn’t be taken lightly.
Kublr, an enterprise-grade Kubernetes platform, helps address the security and governance challenges by setting up some key extensions by default and integrating with your identity management system and RBAC. While you may still have to add extensions that are specific to your needs, the most important frameworks are there. To learn more about how Kublr addresses extensions challenges, schedule a demo.