A stateful application such as a database application requires
persistent storage to preserve all changes to the storage layer. It also
requires that the Kubernetes pod be able to bind to the same volume when it gets
rescheduled on the same or on a different node. In Kubernetes, a
persistent volume can be created manually or dynamically. The dynamic
volume provisioning allows storage volumes to be created on-demand and
automatically. The dynamic provisioning mechanism reduces administrative
overhead since the storage volume will be managed automatically by a
set of predefined rules..
 |
Dynamic Volume Provisioning using NFS CSI - OpenLens dashboard
|
Using NFS CSI driver for Kubernetes: A properly
configured NFS volume in Kubernetes can satisfy persistent volume requirements and
can support a container’s moderate workload (Read/Write). A popular NFS CSI driver
is “csi-driver-nfs” which supports dynamically persistent volume creation, volume
snapshot, volume cloning and volume expansion (references are at the bottom).
Installing NFS CSI plugins in Kubernetes: Before
installing “csi-driver-nfs” in Kubernetes, review the GitHub documentation at https://github.com/kubernetes-csi/csi-driver-nfs.
In this tutorial, we will be using Helm Package Manager
to install the NFS CSI driver.
 |
SQL Server as stateful application in Kubernetes
|
Step#1: Install helm
package manger:
Consult the official Helm Package Manger install process at
https://helm.sh/docs/intro/install/ Login or SSH to the control plane node and then execute the following commands as root:
Switch to root:
# sudo -i
Install Helm packagfe Manager:
# curl https://baltocdn.com/helm/signing.asc | gpg --dearmor | sudo tee /usr/share/keyrings/helm.gpg > /dev/null
# apt-get install apt-transport-https --yes
# echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/helm.gpg] https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list
# apt-get update
# apt-get install helm
Step#2: Using Helm to
install NFS CSI:
# helm repo add csi-driver-nfs https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/charts
Search latest chart version:
# helm search repo -l csi-driver-nfs
Use the latest version of NFS CSI driver or use the specific version you like:
# helm install csi-driver-nfs csi-driver-nfs/csi-driver-nfs --namespace kube-system --version v4.5.0
Once installed, verify that NFS CSI is running on all nodes:
# kubectl get pod -n kube-system -o wide | grep nfs
Step#3: Creating a StorageClass
for Dynamic Volume Provisioning:
Once we have installed NFS CSI driver in Kubernetes, the
next step is to create:
- A Storage Class (SC)
- A Persistent Volume Claim (PVC)
- A pod which will claim the PVC
To create a storage class, we need to have a NFS share somewhere in the network. If you don’t have one, then you’ll need to install and configure a NFS share and then map the NFS client (control plane) root user to the NFS server root user. In this example, we are using QNAP NFS v4.1. The required permissions have been granted to access the share as root from the Kubernetes control plane.
NFS Server IP: 192.168.0.25 NFS Share: kubedata Create a storage class:
Following is the yaml for the storage class object. Save
it as "nfs_sc.yaml".
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: nfs-data
provisioner: nfs.csi.k8s.io
parameters:
server: 192.168.0.25
share: /kubedata
reclaimPolicy: Retain
allowVolumeExpansion: true
volumeBindingMode: Immediate
mountOptions:
- hard
- nfsvers=4.1
Now apply the yaml:
# kubectl apply -f nfs_sc.yaml
Example #1: A Pod using Dynamic Volume Provisioning:
When we use the dynamic volume provisioning method, we
don’t need to manually create the Persistent Volume (PV) in advance. Instead, when
the Persistent Volume Claim (PVC) is created, the required PV will also be automatically
created and bound to the PVC. When a pod is created by referring to the PVC
name, the required storage will be attached with the pod.
When creating a stateful pod, the Persistent Volume Claim
(PVC) name must be provided in the specification section of the storageClassName.
If we make the storage class the default storage in the Kubernetes cluster, then
the storageClassName is not required. A PV will automatically be created from
the default storage of Kubernetes as per the PVC and the required storage will
be attached to the container.
Create a Persistent Volume Claim (PVC) yaml. Save the
file as "nfs_pvc.yaml":
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: nfs-srvsql01-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
storageClassName: nfs-data
Apply the yaml to create the PVC (nfs-srvsql01-pvc):
# kubectl apply -f nfs-pvc.yaml Creating a deployment and claiming the PVC: In this step, we will create a SQL Server container:
- Create a SQL Server deployment yaml file (sql1.yaml)
- Create a Nodeport service to connect the SQL Server Instance from the network
The "sql1.yaml" file contains the following definition:
apiVersion: apps/v1
kind: Deployment
metadata:
name: srvsql01
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: srvsql01
template:
metadata:
labels:
app: srvsql01
spec:
terminationGracePeriodSeconds: 0
hostname: srvsql01
securityContext:
fsGroup: 10001
containers:
- name: srvsql01
image: mcr.microsoft.com/mssql/server:2022-latest
ports:
- containerPort: 1433
env:
- name: MSSQL_SA_PASSWORD
value: "FantasticPassword"
- name: MSSQL_PID
value: "QQQQQ-PPPPPP-DDDDD-WWWWW-RRRRR"
- name: ACCEPT_EULA
value: "Y"
- name: MSSQL_TCP_PORT
value: "1433"
- name: MSSQL_AGENT_ENABLED
value: "true"
resources:
requests:
memory: 4Gi
cpu: '2'
limits:
memory: 4Gi
volumeMounts:
- name: srvsql01-vol
mountPath: /var/opt/mssql
subPath: srvsql01
volumes:
- name: srvsql01-vol
persistentVolumeClaim:
claimName: nfs-srvsql01-pvc
---
apiVersion: v1
kind: Service
metadata:
name: srvsql01-svc
spec:
type: NodePort
selector:
app: srvsql01
ports:
- name: srvsql01
port: 1433
nodePort: 31433
targetPort: 1433
protocol: TCP
Create the deployment:
# kubectl apply -f sql1.yaml
Example #2: Creating StatefulSet
replica using Dynamic Volume Provisioning.
A stateful deployment of a container is slightly
different than a simple deployment. The basic steps are:
- Create a service definition
- Create a StatefulSet definition with volumeClaimTemplates
In the stateful definition, the critical part is the volumeClaimTemplates
section. This is the section where we define the PersistentVolumeClaim. When a
StatefulSet needs to create a pod replica, it uses the volumeClaimTemplates
definition to create a PVC, and then a PV will automatically be created with the
required volume for the pod.
Following is the StatefulSet definition. Save it as
"sql2.yaml" and then apply it using kubectl.
# First define the service
apiVersion: v1
kind: Service
metadata:
name: srvsql03-svc
spec:
type: NodePort
selector:
app: srvsql03
ports:
- name: srvsql03
port: 3433
nodePort: 31033
targetPort: 3433
protocol: TCP
---
# Create the stateful replica
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: srvsql03
spec:
replicas: 1
selector:
matchLabels:
app: srvsql03
serviceName: "srvsql03-svc"
template:
metadata:
labels:
app: srvsql03
spec:
terminationGracePeriodSeconds: 10
hostname: srvsql03
securityContext:
fsGroup: 10001
containers:
- name: srvsql03
image: mcr.microsoft.com/mssql/server:2022-latest
ports:
- containerPort: 3433
env:
- name: MSSQL_SA_PASSWORD
value: "YourPasswordHere"
- name: MSSQL_PID
value: "ABCDE-XYZXY-ZZZZZ-GGGGG-XZZZZ"
- name: ACCEPT_EULA
value: "Y"
- name: MSSQL_TCP_PORT
value: "3433"
- name: MSSQL_AGENT_ENABLED
value: "true"
resources:
requests:
memory: 4Gi
cpu: '2'
limits:
memory: 4Gi
volumeMounts:
- name: nfs-srvsql03-pvc
mountPath: /var/opt/mssql
subPath: srvsql03
# Dynamic volume claim goes here
volumeClaimTemplates:
- metadata:
name: nfs-srvsql03-pvc
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "nfs-data"
resources:
requests:
storage: 6Gi
Create the stateful replica:
# kubectl apply -f sql2.yaml
 |
Using NFS CSI in Kubernetes cluster
|
References:
Dynamic Volume Provisioning:
https://kubernetes.io/docs/concepts/storage/dynamic-provisioning/
StatefulSets:
https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/
NFS CSI driver for Kubernetes:
https://github.com/kubernetes-csi/csi-driver-nfs
Kubernetes Container Storage Interface (CSI) Documentation:
https://kubernetes-csi.github.io/docs/drivers.html
Helm Package Manager:
https://helm.sh/docs/intro/install/
