Kubernetes ABCs

Context

Kubernetes was created by Google based on lessons learned running containers at scale for a lot of years and it was donated to the community as an open-source project. It runs on any cloud or on-premises datacenter and abstracts the underlying infrastructure (this allows you to build hybrid clouds, as well as migrate on and off the cloud and between different clouds).

Overview

Kubernetes is like any other cluster, a bunch of machines that host applications, but also acts as an orchestrator, taking care of deploying and managing them. We call these machines “nodes”, and they can be physical servers, virtual machines, cloud instances, Raspberry Pis, and more.

Control plane nodes are servers where the control plane components run. Under-the-hood, the control-plane comprises several services, including the API server that exposes a public REST interface to the cluster and where all the scheduling decisions are made, the cluster store (based on etcd) that persistently stores the entire configuration and state of the cluster, the controller manager, that implements all the background controllers that monitor cluster components and respond to events (control loop) and the scheduler, that watches the API server for new work tasks and assigns them to appropriate healthy worker nodes.

Worker nodes, on the other hand, are servers where user applications run. Every node runs a service called the kubelet that registers it with the cluster and communicates with the API server (it watches the API server for new work tasks and maintains a reporting channel). They also have a container runtime and the kube-proxy service: the container runtime is responsible for all container-related operations (such as pulling images and starting and stopping containers) and the kube-proxy is responsible for local networking on the node.

Kubernetes objects

To represent the state of a kubernetes cluster there exist entities called “kubernetes objects”, usually represented in JSON or YAML files; once they are created, the cluster does its best to ensure it exists as defined (this is known as the cluster’s “desired state.”).

Namespaces

Namespaces are a native way to divide a single Kubernetes cluster into multiple virtual clusters. They are designed as an easy way to apply quotas and policies to groups of objects. They’re not designed for strong workload isolation (“If you need strong workload isolation, the current method is to use multiple clusters).

Pods

Pods are the basic building-block for running one or more containers and they are also the minimum unit of scaling; they are mortal and immutable: they’re created, they live, and they are killed and replaced if configuration needs to be updated.

The .metadata section is where you attach things such as names, labels, annotations, and a Namespace. The name helps you identify the object in the cluster (the pod hostname it’s inherited from it), and the labels let you create loose couplings with other objects. Annotations can help integrate with 3rd-party tools and services.

The .spec section is where you define the containers the Pod will run (Pod template). The example is defining a single-container Pod based on the nigelpoulton/k8sbook:1.0 image, calling the container hello-ctr and exposing it on port 8080. Remember that the created Pod will be only accessible through its IPs from inside the cluster; to reach it from the outside you will need to create another Kubernetes object called Service (explained later).

Deployments

Deployments are higher-level Kubernetes objects that wrap around Pods (they only manage a single Pod template, but can manage several replicas) and augment them with self-healing, scaling, and updates/rollbacks. In fact, behind-the-scenes, Deployments manage another object called ReplicaSet to do the self-healing and scaling, and only add updates/rollbacks on top.

After being created, deployments run as a watch loop, constantly observing the cluster, making sure the observed state matches the desired state; if it doesn’t match, the process of reconciliation brings them back together. For example, when the desired state is 10 replicas, but only 8 are present, it brings up 2 more. The same applies when an update is performed, i.e: when a new image version is set on the pod template, Kubernetes creates a new replica running the new version and terminates an existing one running the old version; as Kubernetes increases the number of Pods in the new ReplicaSet (with the new version of the image) it decreases the number of Pods in the old ReplicaSet (with the old version of the image) — their configurations still exist on the cluster, making them a great option for reverting to previous versions (rollbacks).

Services

Services add stable networking and basic load-balancing for a set of pods, as replacement, updates and scaling cause a lot of churn, making Pods ips unreliable.

On the front end, they provide a stable DNS name that’s automatically registered with the cluster DNS, as well as a stable virtual IP. On the back end, they load-balance traffic across a dynamic set of Pods that match a label selector.

By default, Services are of type “clusterIP”, that means they cannot be reached from outside; however, if needed, you can change their type to expose elements of your application:

• NodePort type: It enables external access via a dedicated port on every cluster node.
• LoadBalancer type: It makes external access even easier by integrating with an internet-facing load-balancer on your underlying cloud platform.

In the previous example, “selector” is the list of labels the Service looks for when building its list of Pods to send traffic to, “port” is the port the Service listens on inside the cluster, “nodePort” is the cluster-wide port that can be used for external access and “targetPort” is the port the app is listening on.

Ingress

As seen above, Services allow you to expose your applications to the outside world, but they have limitations: NodePorts only work on high port numbers (30000–32767) and require knowledge of node names or IPs, and LoadBalancer requires 1.1 mapping between the cloud services. and load balancers (and can be expensive and finite). Therefore, Ingress object solves this problem allowing multiple Kubernetes applications and services to be exposed through a single cloud load balancer; they are basically sets of rules that govern how incoming HTTP/HTTPS traffic is routed.

When the request hits the load balancer, the Ingress object is watching, and routes the traffic to the appropriate Service, based on the host or the path based:

Other

ConfigMap/Secrets

ConfigMaps and Secrets are the Kubernetes native way of decoupling applications and config data; both can be injected into containers at run-time via various constructs, with volumes being the preferred method, as they allow updates to eventually be reflected in running containers (other ways are using environment variables or arguments on the startup command).

ConfigMaps are designed for application configuration parameters and even entire configuration files, whereas Secrets are designed for sensitive configuration data (Despite being designed for sensitive data, Kubernetes does not encrypt Secrets. It merely obscures them as base-64 encoded values that can easily be decoded. Fortunately, it’s possible to configure encryption-at-rest with EncryptionConfiguration objects, and most service meshes encrypt network traffic)

StatefulSets

StatefulSets, as Deployments, can self-heal, scale up and down, and perform updates, but they create and manage applications that need to persist state. Each Pod replica spawned by a StatefulSet gets a predictable and persistent name, DNS hostname, and a unique set of volumes, that stay with the Pod for its entire lifecycle.

StatefulSets create one Pod at a time, and always wait for previous Pods to be running and ready before creating the next (This is different from Deployments that use a ReplicaSet controller to start all Pods at the same time, causing potential race conditions).

If you want to go deeper into this topic, you can read the book on which this post was based: “The Kubernetes Book” by Nigel Poulton