Lab 4 - Kubernetes Workloads
Introduction¶
Welcome to the lab 4. In this session, the next topics are covered:
- Getting familiar with basic Kubernetes resources;
- Deployment of NGINX server on Kubernetes;
- Setup of Kubernetes workloads for ghostfolio application;
- Configuration of the application secrets.
Workload basics¶
In terms of Kubernetes, a manifest is a .yaml file with description of Kubernetes-managed resource. Typical resources are Pod (container group), Deployment (management for stateless Pods like web services), ConfigMap (set of configuration data), Secret (set of encrypted key-value pairs) StatefulSet (management for stateful Pods, like database services).
Pod workload¶
A Pod is a minimal unit of workload in Kubernetes, which represents a set of containers with common storage and network resources.
An example manifest file for Pod with NGINX server container looks like this:
apiVersion: v1
kind: Pod
metadata:
name: nginx
labels:
name: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
resources:
requests:
memory: "128Mi"
cpu: "300m"
limits:
memory: "256Mi"
cpu: "500m"
ports:
- containerPort: 80
name: http
The major sections are:
apiVersion
- a version of Kubernetes API to use;kind
- a type of Kubernetes resource,Pod
in this case;metadata
describes Pod name and labels for filtering;spec
describes a specification for containers, volumes, etc.;containers
describes settings for containers within a Pod and includes container images, exposed ports and computing resources.
Complete
To deploy this Pod on Kubernetes cluster, put this manifest into a file (for example pod-nginx.yaml
) and run:
export KUBECONFIG=/etc/kubernetes/admin.conf
kubectl apply -f pod-nginx.yaml
# pod/nginx created
You can get the Pod info:
kubectl get pod nginx
# NAME READY STATUS RESTARTS AGE
# nginx 1/1 Running 0 2m43s
Feel free to login into the Pod's container and explore it:
kubectl exec -it nginx -- /bin/bash
# root@nginx:/#
Use this command to get detailed info about the Pod.
kubectl describe pod nginx
# Name: nginx
# Namespace: default
# ...
# Status: Running
# IP: 10.0.1.244
# ...
It serves the same purpose as nerdctl inspect container
, but also describes Kubernetes-related metadata.
Validate
Using IP field, you can access the server welcome page:
curl 10.0.1.244
#<a href="http://nginx.com/">nginx.com</a>.</p>
#
#<p><em>Thank you for using nginx.</em></p>
#...
Although accessing the Pod via browser isn't possible now, the next lab explain the way to do it.
Deployment workload¶
One of the most powerful features of Kubernetes is Pod lifecycle management. It covers many cases, for example: Pod gets restarted when an application container crashes with error. For this, Kubernetes requires a higher-level structure Deployment to manage stateless Pods.
If the NGINX Pod is removed, Kubernetes won't recreate it automatically:
kubectl delete pod nginx
# pod "nginx" deleted
kubectl get pod nginx
# Error from server (NotFound): pods "nginx" not found
Complete
Let's create a Deployment manifest:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
resources:
limits:
memory: "256Mi"
cpu: "500m"
ports:
- containerPort: 80
name: http
And create Kubernetes resource:
kubectl apply -f deployment-nginx.yaml
# deployment.apps/nginx created
Verify
After creation, you can find both the Deployment and a linked Pod:
kubectl get deployment nginx
# NAME READY UP-TO-DATE AVAILABLE AGE
# nginx 1/1 1 1 35s
kubectl get pod -l app=nginx
# NAME READY STATUS RESTARTS AGE
# nginx-74547bd6d7-smgmk 1/1 Running 0 51s
kubectl describe deployment nginx
As you can see from the last command, the Deployment doesn't include any specific information about containers, rather the Pod template and availability info.
Deployment of Ghostfolio application¶
Ghostfolio is a chosen app as a primary use case for this course.
The app's architecture consists of:
- frontend (Angular);
- backend (NetsJS);
- database (PostgreSQL);
- caching system (Redis).
A docker-compose configuration provided by the development team is in the repository: link. Application deployment consists of 3 containers: Redis, PostgreSQL and the application (frontend + backend).
The Kubernetes setup requires 1 StatefulSet for database and 2 Deployments for cache and the application.
Redis¶
Redis is an in-memory database, which has basic authentication capabilities. In this lab, we are going to use password-base auth. In Kubernetes, the proper way to store static passwords, tokens, etc. is Secret resource.
Info
A Secret represents a set of key-value pairs, where a key is an alias and a value is base64-encoded secret. This resource is namespace-based, therefore a Secret instance can't be shared between namespaces.
Complete
A manifest for a Redis credentials looks like this:
apiVersion: v1
kind: Secret
metadata:
name: redis-secret
type: Opaque
data:
password: cmVkaXMtcGFzc3dvcmQ= # base64 encoded "redis-password"
Opaque type means the data in the secret is generic and doesn't relate to Kubernetes cluster. The values in data
section must be encoded by a user, otherwise the cluster raises the error: error decoding from json: illegal base64 data
.
Now, let's apply the manifest:
kubectl apply -f secret-redis.yaml
# secret/redis-secret created
Verify
Validate the Secret has been successfully created:
kubectl get secret redis-secret
# NAME TYPE DATA AGE
# redis-secret Opaque 1 10s
The next step is Deployment of Redis. For now, we create a storage without persistence and with a single replica.
Complete
Create a Deployment with Redis setup and apply the manifest:
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis
spec:
selector:
matchLabels:
app: redis
replicas: 1
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: bitnami/redis:6.2.13
env:
- name: REDIS_PASSWORD
valueFrom:
secretKeyRef:
name: redis-secret
key: password
resources:
requests:
memory: 128Mi
cpu: 100m
limits:
memory: 256Mi
cpu: 200m
ports:
- containerPort: 6379
name: redis-port
As you can notice, the redis-deployment
references the redis-secret
in the manifest. This approach is helpful, when an application needs to share the credentials with a database service.
Verify
If you try to get a Pod by the label, the output should be similar to this one:
kubectl get pods -l app=redis
# NAME READY STATUS RESTARTS AGE
# redis-7cfc5c8d5c-x775z 1/1 Running 0 3m23s
Also, please check the logs to ensure the process has successfully started:
kubectl logs redis-7cfc5c8d5c-x775z
# ... Welcome to the Bitnami redis container
# ...* Ready to accept connections
Currently, the Redis instance is not properly accessible via network. If an application running in the cluster needs to request Redis, it should know the Pod's hostname or IP. This is not a reliable approach, because pods can be recreated and the hostname changes after this.
To encapsulate one or set of Pods, a Service resource is recommended. It works as a single-point access to Pods and provides load balancing. In the next lab, we are going to dive deeper into Kubernetes networking topic.
Complete
For now, let's create a simple service for Redis server:
apiVersion: v1
kind: Service
metadata:
name: redis
spec:
selector:
app: redis
ports:
- port: 6379
targetPort: 6379
PostgreSQL¶
PostgreSQL is a free object-relational database system. Many IT companies make use of it due to openness, included features and large community. In this practice, we use this system as a primary data storage for the use-case app.
Complete
First of all, we need to create a Secret with database credentials:
apiVersion: v1
kind: Secret
metadata:
name: postgresql-secret
type: Opaque
data:
username: Z2hvc3Rmb2xpbw== # base64 encoded ghostfolio
database: Z2hvc3Rmb2xpbw== # base64 encoded ghostfolio
postgresPassword: cG9zdGdyZXMtcGFzc3dvcmQ= # base64 encoded postgres-password
kubectl apply -f secret-postgresql.yaml
# secret/postgresql-secret created
This is used for DB initialization and further access by Ghostfolio.
Now, we are ready to deploy a StatefulSet for PostgreSQL. StatefulSet is a management resource as a Deployment, which controls stateful applications only. The key differences between the two:
- StatefulSet creates and removes pods in a strict order;
- Deployment Pods are identical and can be interchanged;
- Deployment assigns random name suffix for Pods, StatefulSet uses sequential numbers;
- Pods managed by a Deployment share the same persistent storage while StatefulSet creates a volume for each replica.
For now, the storage setup for Kubernetes is not covered by this lab, so instead of persistent volumes we use in-memory storage.
Complete
Create a StatefulSet with PostgreSQL setup:
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgresql
spec:
selector:
matchLabels:
app: postgresql
serviceName: postgresql
replicas: 1
template:
metadata:
labels:
app: postgresql
spec:
containers:
- name: postgresql
image: bitnami/postgresql:15.4.0
env:
- name: POSTGRESQL_USERNAME
valueFrom:
secretKeyRef:
name: postgresql-secret
key: username
- name: POSTGRESQL_DATABASE
valueFrom:
secretKeyRef:
name: postgresql-secret
key: database
- name: POSTGRESQL_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql-secret
key: postgresPassword
ports:
- containerPort: 5432
name: postgres-port
volumeMounts:
- name: postgresql-data
mountPath: /bitnami/postgresql
volumes:
- name: postgresql-data
emptyDir: {}
Verify
View logs for postgresql-0 Pod:
kubectl logs postgresql-0
# ...
# ... [1] LOG: database system is ready to accept connections
# ...
The log should contain database system is ready to accept connections
as indication of successful start.
Also, check if you can login into the Pod:
kubectl exec -it postgresql-0 -- bash
# I have no name!@postgresql-0:/$
and check access to the PostgreSQL CLI:
export PGPASSWORD="postgres-password"
psql -U ghostfolio
# psql (15.4)
# Type "help" for help.
# ghostfolio=>
Complete
In this section you need to create a service for PostgreSQL yourself. You can use Redis service as an example.
NB: Please, use postgresql
as a service name
Ghostfolio app¶
Ghostfolio uses a single container for the frontend and the backend and doesn't include any persistent data. Deployment fits these requirements and, in this section, you need to create and apply a manifest yourself.
Complete
You can find a container image for Ghostfolio in docker-compose.yaml.
The app requires the following environment variables:
DATABASE_URL
, format:postgresql://<POSTGRES_USER>:<POSTGRES_PASSWORD>@postgresql:5432/<POSTGRES_DB>?sslmode=prefer
NODE_ENV
, assignproduction
valueREDIS_HOST
, assignredis
REDIS_PORT
, assign6379
REDIS_PASSWORD
, assignpassword
field fromredis-secret
ACCESS_TOKEN_SALT
, assign a random alphanumeric stringJWT_SECRET_KEY
, assign a random alphanumeric string
The first one can be added to the existing postgresql-secret and imported the same way:
...
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: postgresql-secret
key: url
...
You can change the content of secret-postgresql.yaml
file by adding an url
field with base64 encoded URL string as a value.
To apply the changes, use the same kubectl apply -f secret-postgresql.yaml
command:
kubectl apply -f secret-postgresql.yaml
# secret/postgresql-secret configured
If you want to apply secret changes for the Deployment after creation , you need to restart it:
kubectl rollout restart deployment ghostfolio
# deployment.apps/ghostfolio restarted
ACCESS_TOKEN_SALT
and JWT_SECRET_KEY
variables can be in a separate secret too.
NB: please add label app: ghostfolio
to the template of the Ghostfolio pod.
The resource requests and limits could be like this:
resources:
requests:
memory: 512Mi
cpu: 1
limits:
memory: 1024Mi
cpu: 2
Verify
kubectl logs -l app=ghostfolio
# ...
# Listening at http://0.0.0.0:3333
If the massage Listening at http://0.0.0.0:3333
eventually shows, then the application is up and running. You can access the application by the Pod's IP:
curl -I -L 10.0.1.122:3333
# HTTP/1.1 200 OK
# X-Powered-By: Express
# Access-Control-Allow-Origin: *
# Content-Type: text/html; charset=utf-8
# Content-Length: 28290
# ETag: W/"6e82-Hj3f0VuXQ7VokGpxgLIBEvLI1jk"
# Date: Thu, 21 Sep 2023 12:42:59 GMT
# Connection: keep-alive
# Keep-Alive: timeout=5