K8s - DigitalOcean

2023-12-08 (Last Modified: 2024-01-23)

Cloud

Page content

Deploy Sampleapp on Kuberentes …

Prerequisite

Domain “kubbi.xyz”, ns1.digitalocean.com, ns2, ns3 …
Digital Ocean Login

Doku

https://docs.digitalocean.com/products/kubernetes/getting-started/operational-readiness/

Build Kubbi Cluster

Build Cluster with WebGUI or CLI

via CLI

time doctl kubernetes cluster create prod001 --region fra1 --node-pool "size=s-2vcpu-2gb;auto-scale=true;min-nodes=3;max-nodes=5"

FRA1
3 Nodes
2 CPU
2 GB RAM
60 GB Disk
Costs: 54 USD/Mt!

Connecting and managing this cluster

doctl kubernetes cluster kubeconfig save 4375b470-ebe8-4ccb-925a-345df364dfbd

user@mac % doctl kubernetes cluster kubeconfig save 4375b470-ebe8-4ccb-925a-345df364dfbd


Notice: Adding cluster credentials to kubeconfig file found in "/Users/user/.kube/config"
Notice: Setting current-context to do-fra1-k8s-1-28-2-do-0-fra1-1702031438694

kubectl config get-contexts

kubectl config get-contexts

user@mac % kubectl config get-contexts

CURRENT   NAME                                         CLUSTER                                      AUTHINFO                                           NAMESPACE
*         do-fra1-prod-cluster-01                      do-fra1-prod-cluster-01                      do-fra1-prod-cluster-01-admin

kubectl cluster-info

kubectl cluster-info

user@mac % kubectl cluster-info

Kubernetes control plane is running at https://f179692f-aeac-4f9c-af3b-2422897ea578.k8s.ondigitalocean.com
CoreDNS is running at https://f179692f-aeac-4f9c-af3b-2422897ea578.k8s.ondigitalocean.com/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

kubectl version

kubectl version

user@mac % kubectl version

Client Version: v1.28.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.2

kubectl get nodes

kubectl get nodes

user@mac % kubectl get nodes

NAME                    STATUS   ROLES    AGE     VERSION
prod-cluster-01-xa856   Ready    <none>   3m59s   v1.28.2
prod-cluster-01-xa85a   Ready    <none>   4m7s    v1.28.2

Install Nginx Ingress Controller

via GUI / Marketplace / NGINX Ingress Controller

show lbl status

doctl compute load-balancer list --format IP,ID,Name,Status

user@mac % doctl compute load-balancer list --format IP,ID,Name,Status

IP    ID                                      Name                                Status
      55d7381d-9bcd-4f8f-905e-e90efd09f03e    a6554a9aff1a340e986d48431e19cca9    new

no ip yet … wait …

while true; do doctl compute load-balancer list --format IP,ID,Name,Status; sleep 10; done

after a few seconds …

user@mac % while true; do doctl compute load-balancer list --format IP,ID,Name,Status; sleep 10; done

IP    ID                                      Name                                Status
      55d7381d-9bcd-4f8f-905e-e90efd09f03e    a6554a9aff1a340e986d48431e19cca9    new
IP    ID                                      Name                                Status
      55d7381d-9bcd-4f8f-905e-e90efd09f03e    a6554a9aff1a340e986d48431e19cca9    new
IP    ID                                      Name                                Status
      55d7381d-9bcd-4f8f-905e-e90efd09f03e    a6554a9aff1a340e986d48431e19cca9    new
IP    ID                                      Name                                Status
      55d7381d-9bcd-4f8f-905e-e90efd09f03e    a6554a9aff1a340e986d48431e19cca9    new
IP             ID                                      Name                                Status
64.225.93.9    55d7381d-9bcd-4f8f-905e-e90efd09f03e    a6554a9aff1a340e986d48431e19cca9    active
IP             ID                                      Name                                Status
64.225.93.9    55d7381d-9bcd-4f8f-905e-e90efd09f03e    a6554a9aff1a340e986d48431e19cca9    active

Create Backend Application

kubectl create ns backend

user@mac % kubectl create ns backend

namespace/backend created

Create the Deployment and apply

cat << 'EOF' > echo_deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo
  namespace: backend
spec:
  replicas: 1
  selector:
    matchLabels:
      app: echo
  strategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: echo
    spec:
      containers:
        - name: echo
          image: jmalloc/echo-server
          ports:
            - name: http
              containerPort: 8080
          resources:
            requests:
              cpu: 100m
              memory: 50Mi
            limits:
              cpu: 200m
              memory: 100Mi
EOF
kubectl apply -f echo_deployment.yaml

deployment.apps/echo created

Create the Service and Apply

cat << 'EOF' > echo_service.yaml
apiVersion: v1
kind: Service
metadata:
  name: echo
  namespace: backend
spec:
  ports:
    - name: http
      port: 80
      targetPort: 8080
  selector:
    app: echo
EOF
kubectl apply -f echo_Service.yaml

service/echo created

Verify Deployment

kubectl get deployments -n backend

user@mac % kubectl get deployments -n backend

NAME   READY   UP-TO-DATE   AVAILABLE   AGE
echo   1/1     1            1           81s

Verify Services

kubectl get services -n backend

user@mac % kubectl get services -n backend

NAME   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
echo   ClusterIP   10.245.228.215   <none>        80/TCP    43s

Configure DNS

doctl compute domain records list kubbi.xyz

user@mac % doctl compute domain records list kubbi.xyz

ID            Type    Name    Data                    Priority    Port    TTL     Weight
1715497047    SOA     @       1800                    0           0       1800    0
1715497048    NS      @       ns1.digitalocean.com    0           0       1800    0
1715497049    NS      @       ns2.digitalocean.com    0           0       1800    0
1715497050    NS      @       ns3.digitalocean.com    0           0       1800    0

add Wildcard Pointing to the Cluster

lbip="64.225.93.9"
doctl compute domain records create "kubbi.xyz" --record-type A --record-data ${lbip} --record-ttl 60 --record-name "*"

confirm Wildcard Record

user@mac % doctl compute domain records list kubbi.xyz

ID            Type    Name    Data                    Priority    Port    TTL     Weight
1715497047    SOA     @       1800                    0           0       1800    0
1715497048    NS      @       ns1.digitalocean.com    0           0       1800    0
1715497049    NS      @       ns2.digitalocean.com    0           0       1800    0
1715497050    NS      @       ns3.digitalocean.com    0           0       1800    0
1715597396    A       *       64.225.93.9             0           0       60      0

Install Cert Manager

via GUI / Marketplace / Cert-Manager

Check Cert Manager

kubectl get all -n cert-manager

user@mac % kubectl get all -n cert-manager

NAME                                           READY   STATUS    RESTARTS   AGE
pod/cert-manager-7b9cbf5b8d-p2bwm              1/1     Running   0          13s
pod/cert-manager-cainjector-6d5f558c69-m79p9   1/1     Running   0          13s
pod/cert-manager-webhook-bd76f6cf9-48bwv       1/1     Running   0          13s

NAME                           TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/cert-manager-webhook   ClusterIP   10.245.93.140   <none>        443/TCP   13s

NAME                                      READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/cert-manager              1/1     1            1           13s
deployment.apps/cert-manager-cainjector   1/1     1            1           13s
deployment.apps/cert-manager-webhook      1/1     1            1           13s

NAME                                                 DESIRED   CURRENT   READY   AGE
replicaset.apps/cert-manager-7b9cbf5b8d              1         1         1       13s
replicaset.apps/cert-manager-cainjector-6d5f558c69   1         1         1       13s
replicaset.apps/cert-manager-webhook-bd76f6cf9       1         1         1       13s

Inspect the Cert Manager

kubectl get crd -l app.kubernetes.io/name=cert-manager

user@mac % kubectl get crd -l app.kubernetes.io/name=cert-manager

NAME                                  CREATED AT
certificaterequests.cert-manager.io   2023-12-08T06:18:47Z
certificates.cert-manager.io          2023-12-08T06:18:47Z
challenges.acme.cert-manager.io       2023-12-08T06:18:47Z
clusterissuers.cert-manager.io        2023-12-08T06:18:48Z
issuers.cert-manager.io               2023-12-08T06:18:47Z
orders.acme.cert-manager.io           2023-12-08T06:18:47Z

Create Kubernetes Secret

via GUI / API: https://cloud.digitalocean.com/account/api/tokens

mytoken
90 days
read/write

dop_v1_60f1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

DO_API_TOKEN="dop_v1_60f1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

kubectl create secret generic "digitalocean-dns" \
  --namespace backend \
  --from-literal=access-token="$DO_API_TOKEN"

user@mac % DO_API_TOKEN="dop_v1_60f1exxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

kubectl create secret generic "digitalocean-dns" \
  --namespace backend \
  --from-literal=access-token="$DO_API_TOKEN"
secret/digitalocean-dns created

Configure Issuer CRD and apply

adapt email

cat << 'EOF' > cert-manager-wcard-issuer.yaml
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: letsencrypt-dev-wcard
  namespace: backend
spec:
  acme:
    email: certs@user.net
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-dev-wcard-private
    # List of challenge solvers that will be used to solve ACME challenges for the matching domains.
    solvers:
    # Use the DigitalOcean DNS API to manage DNS-01 challenge records.
      - dns01:
          digitalocean:
            # Kubernetes secret that contains the DO API token.
            # Must be in the same namespace as the Issuer CRD.
            tokenSecretRef:
              name: digitalocean-dns
              key: access-token
EOF
kubectl apply -f cert-manager-wcard-issuer.yaml

issuer.cert-manager.io/letsencrypt-dev-wcard created

verify issuer

kubectl get issuer letsencrypt-dev-wcard -n backend

user@mac % kubectl get issuer letsencrypt-dev-wcard -n backend

NAME                    READY   AGE
letsencrypt-dev-wcard   True    25s

Configure Certificate CRD and apply

cat << 'EOF' > cert-manager-wcard-certificate.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: kubbi.xyz
  namespace: backend
spec:
  secretName: kubbi.xyz
  issuerRef:
    name: letsencrypt-dev-wcard
    kind: Issuer
    group: cert-manager.io
  commonName: "*.kubbi.xyz"
  dnsNames:
    - "kubbi.xyz"
    - "*.kubbi.xyz"
EOF
kubectl apply -f cert-manager-wcard-certificate.yaml

certificate.cert-manager.io/kubbi.xyz created

verify status

kubectl get certificate kubbi.xyz -n backend

user@mac % kubectl get certificate kubbi.xyz -n backend

NAME        READY   SECRET      AGE
kubbi.xyz   False   kubbi.xyz   24s

wait 2:30min

user@mac % kubectl get certificate kubbi.xyz -n backend

NAME        READY   SECRET      AGE
kubbi.xyz   True    kubbi.xyz   3m39s

Describe Secret

kubectl describe secret kubbi.xyz -n backend

user@mac % kubectl describe secret kubbi.xyz -n backend

Name:         kubbi.xyz
Namespace:    backend
Labels:       controller.cert-manager.io/fao=true
Annotations:  cert-manager.io/alt-names: *.kubbi.xyz,kubbi.xyz
              cert-manager.io/certificate-name: kubbi.xyz
              cert-manager.io/common-name: *.kubbi.xyz
              cert-manager.io/ip-sans:
              cert-manager.io/issuer-group: cert-manager.io
              cert-manager.io/issuer-kind: Issuer
              cert-manager.io/issuer-name: letsencrypt-dev-wcard
              cert-manager.io/uri-sans:

Type:  kubernetes.io/tls

Data
====
tls.crt:  5518 bytes
tls.key:  1675 bytes

create Host and apply

cat << 'EOF' > wildcard-host.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-backend
  namespace: backend
spec:
  tls:
    - hosts:
        - "*.kubbi.xyz"
      secretName: kubbi.xyz
  rules:
    - host: echo.kubbi.xyz
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: echo
                port:
                  number: 80
  ingressClassName: nginx
EOF
kubectl apply -f wildcard-host.yaml

ingress.networking.k8s.io/ingress-backend created

show ingress

kubectl get ingress -n backend

user@mac % kubectl get ingress -n backend

NAME              CLASS   HOSTS            ADDRESS       PORTS     AGE
ingress-backend   nginx   echo.kubbi.xyz   64.225.93.9   80, 443   58s

and here we are …

curl -Li http://echo.kubbi.xyz/

user@puffy202 $ curl -Li http://echo.kubbi.xyz/
HTTP/1.1 308 Permanent Redirect
Date: Fri, 08 Dec 2023 11:16:00 GMT
Content-Type: text/html
Content-Length: 164
Connection: keep-alive
Location: https://echo.kubbi.xyz

HTTP/2 200
date: Fri, 08 Dec 2023 11:16:00 GMT
content-type: text/plain
content-length: 334
strict-transport-security: max-age=15724800; includeSubDomains

Request served by echo-67dbbdb49d-dt9f9

GET / HTTP/1.1

Host: echo.kubbi.xyz
Accept: */*
User-Agent: curl/8.4.0
X-Forwarded-For: 10.244.0.87
X-Forwarded-Host: echo.kubbi.xyz
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Scheme: https
X-Real-Ip: 10.244.0.87
X-Request-Id: 1480dae4d90eab8df39db4683c52404b
X-Scheme: https

Any Comments ?

sha256: 2c4814af7931dcb4ff1080d8d406e8ffe603a213d0ed3b34d00defdcdcebb428