antony@notes:~/learning-notes$ cat "CKA.md"

CKA

2022-10-23· learning-notes ·CKA

CKA

:::spoiler 目錄 [TOC] :::

參考文件 1.https://zhuanlan.zhihu.com/p/564737349 2.https://github.com/kubernetes-csi/external-snapshotter 3.https://github.com/kubernetes-csi/csi-driver-host-path 4.https://github.com/kubernetes-csi/csi-driver-host-path/blob/master/examples/csi-storageclass.yaml 5.https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/ha-topology/ 6.https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/ha-topology/ 7.https://rafay.co/the-kubernetes-current/etcd-kubernetes-what-you-should-know/

考試環境圖

左上角有一個筆記本可以使用，方便寫 yaml 檔（Applications 的 Mousepad）

1. Scale

題型 1：Scale the deployment presenation to 3 pods 題型 2：Scale the deployment loadbalabcer to 6 pods

環境準備

$ kubectl create deploy presenation --image=quay.io/cloudwalker/nginx

$ kubectl create deploy loadbalabcer --image=quay.io/cloudwalker/nginx

$ kubectl get all

解答

題型 1

$ kubectl scale deploy presentaion –replicas=3

$ kubectl get all

題型 2

$ kubectl scale deploy loadbalabcer –replicas=6

$ kubectl get all

2. NetworkPolicy （7%）

Create new NetworkPolicy named allow-port-from-namespace that allows Pods in the existing namespace internal to connect to port 9000 of other Pods in the same namespace.

Ensure that the new NetworkPolicy:

does not allow access to Pods not listening on port 9000
does not allow access from Pods not in namespace internal

解答

K8S DOC 查詢關鍵字 : networkpolicy
找 Network Policies | Kubernetes 這篇
用第一個 YANL 檔

檢查 namespace，如果沒有這個 namespace 就直接 create $ k get namespace

看 Namespace 的標籤

$ k get ns internal --show-labels
NAME       STATUS   AGE     LABELS
internal   Active   3m27s   kubernetes.io/metadata.name=internal

編輯 YAML 檔

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-port-from-namespace
  namespace: internal
spec:
  podSelector: {}                                         # 新增 {}
#    matchLabels:
#      role: db
  policyTypes:
    - Ingress
  ingress:
    - from:
#        - ipBlock:                                       # remove
#            cidr: 172.17.0.0/16                          # remove
#            except:                                      # remove
#              - 172.17.1.0/24                            # remove
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: internal
#        - podSelector:                                   # remove
#            matchLabels:                                 # remove
#              role: frontend                             # remove
      ports:
        - protocol: TCP
          port: 9000                                      # port number 改成 9000
#  egress:                                                # remove
#    - to:                                                # remove
#        - ipBlock:                                       # remove
#            cidr: 10.0.0.0/24                            # remove
#      ports:                                             # remove
#        - protocol: TCP                                  # remove
#          port: 5978                                     # remove

驗證

此功能在 flannel 無法成功
因此使用 Calico ，它的功能比較多

撰寫 yaml

# 建立 bx pod 並且架網站 port 9000
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: bx
  name: bx
  namespace: internal
spec:
  containers:
  - image: quay.io/cloudwalker/busybox
    name: bx
    command:
      - /bin/sh
      - -c
      - |
        mkdir www
        echo hi > www/index.html
        busybox httpd -p 9000 -h www -f
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

驗證同樣在 internal namespace 的 pod 可以透過 9000 port 連線

$ kubectl run alp -it --image=quay.io/cloudwalker/alpine -n internal

# nc -w 1 10.244.190.72 9000;echo $?
0

在 internal namespace 以外都無法透過 9000 port 連線

$ kubectl run alp -it --image=quay.io/cloudwalker/alpine

# nc -w 1 10.244.190.72 9000;echo $?
1

2.2 NetworkPolicy

Create a new NetworkPolicy named allow-port-from-namespace in the existing namespace echo. Ensure that the new NetworkPolicy allows Pods in namespace my-app to connect to port 9000 of Pods in namespace echo.

Further ensure that the new NetworkPolicy:

• does not allow access to Pods, which don’t listen on port 9000 • does not allow access from Pods, which are not in namespace my-app

解答

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-port-from-namespace
  namespace: echo
spec:
  podSelector: {}                                         
  policyTypes:
    - Ingress
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: my-app
      ports:
        - protocol: TCP
          port: 9000

3. Multi-container

題型 1: Create a pod named kucc8 with a single app container for each of the following images runing inside(there may be between 1 and 4 images specified): nginx + memcached

題型 2: Create a pod named kucc1 with a single app container for each of the following images runing inside(there may be between 1 and 4 images specified): nginx + redis + memcached + consul

解答

題型 1:

產生 YAML 檔 $ export do="--dry-run=client -o yaml"

$ kubectl run kucc8 –image=nginx $do > pod.yaml 編輯 YAML 檔

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: kucc8
  name: kucc8
spec:
  containers:
  - image: nginx
    name: kucc8
    resources: {}
  - image: memcached                    # add and change
    name: kucc8-1                       # add
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

產生 pod

$ kubectl apply -f pod.yaml

題型 2:

產生 YAML 檔

$ kubectl run kucc1 –image=nginx $do > pod2.yaml

編輯 YAML 檔

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: kucc1
  name: kucc1
spec:
  containers:
  - image: nginx
    name: kucc1
    resources: {}
  - image: redis                        # add and change
    name: kucc1-1                       # add
  - image: memcached                    # add and change
    name: kucc1-2                       # add
  - image: consul                       # add and change
    name: kucc1-3                       # add
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

產生 pod

$ kubectl apply -f pod2.yaml

4. CPU

From the pod label name=cpu-loader,find pods running high CPU workloads and write the name of the pod consuming most CPU to the file /opt/KUTR00401/KURT00401.txt (while alreay exists).

環境準備

$ kubectl run nginx  --image=quay.io/cloudwalker/nginx --labels=name=cpu-loader

$ kubectl run nginx2  --image=quay.io/cloudwalker/nginx --labels=name=cpu-loader

$ k run alpine1 --image=quay.io/cloudwalker/nginx --labels=name=cpu-loader -- yes > /dev/null

解答

$ kubectl top pod -l “name=cpu-loader” –sort-by=cpu -A

NAME      CPU(cores)   MEMORY(bytes)
alpine1   9m           0Mi
nginx     0m           6Mi
nginx2    0m           2Mi

$ echo “alpine1” > /opt/KUTR00401/KURT00401.txt

5. NodeSelector

Schedule a pod as follows:

name: nginx-kusc00401
Image: nginx
Node selector: disk=spinning

解答

產生 YAML 檔

$ kubectl get nodes --show-labels | grep "disk=spinning"

$ kubectl label node w1 disk=spinning $ kubectl run nginx-kusc00401 –image=nginx –dry-run=client -o yaml > pod.yaml

編輯 YAML 檔

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx-kusc00401
  name: nginx-kusc00401
spec:
  containers:
  - image: nginx
    name: nginx-kusc00401
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
  nodeSelector:                         # add
    disk: spinning                      # add
status: {}

產生 pod

$ kubectl apply -f pod.yaml 檢查

$ kubectl get pod -o wide

6. Kubernetes 升級

Given an existing Kubernetes cluster running version 1.24.1, upgrade all of Kubernetes control plane and node components on the master node only to version 1.24.2

You are also expected to upgrade kubelet and kubectl on the master node.

Be sure to drain the master node before upgrading it and uncordon it after the upgrade. Do not upgrade the worker nodes, etcd, the container manager, the CNI plugin, the DNS service or any other addons.

考試環境 3台機器:1台 master 2台 worker

drain：進入維護模式，會把這台 nodes 上的 pod 趕到別台 node 上，有人照顧(例如:deployment)的 pod 才會在別台 node 產生，沒人照顧的 pod 就會被刪除。
uncordon：解除維護模式，之後就可以在這台 node 產生 pod，但被趕走的 pod 並不會在回到這台 node 上。

解答文件1 文件2

K8S DOC 查詢關鍵字 : upgrading kubeadm
找 Upgrading kubeadm clusters | Kubernetes 這篇對 Linux 套件 kubeadm 進行升級

# ssh 連進 control plane node
$ ssh m1

# 提權，因為考試的環境使用者沒有 sudo 的權限
$ sudo -i

# 這是 ubuntu 的更新套件清單命令
$ apt update

# 看當前主機可以升級 kubeadm 的那些版本
$ apt-cache madison kubeadm | grep 1.24.2
   kubeadm |  1.24.2-00 | https://apt.kubernetes.io kubernetes-xenial/main amd64 Packages

# 將 kubeadm 升級至 1.24.2 版
# unhold ，因為現在要升級，解除鎖定
# hold ，讓系統 update 或 upgrade 的時候，忽略 hold 的套件，以後系統在 update upgrade 的時候就不會升級這些套件(鎖定版本)
# 這裡只是針對套件做升級而已，並沒有升級 kubernetes
# alpine 沒有鎖定版本的功能，只要 upgrade 就全部升級了
$ apt-mark unhold kubeadm && \
  apt-get update && apt-get install -y kubeadm=1.24.2-00 && \
  apt-mark hold kubeadm

# 檢查下載下來的 kubeadm 版本
$ kubeadm version

# Drain control-plane node
$ kubectl drain m1 --ignore-daemonsets 

# 已成功 drain m1
$ kubectl get nodes
NAME   STATUS                     ROLES           AGE   VERSION
m1     Ready,SchedulingDisabled   control-plane   51m   v1.24.1
w1     Ready                      <none>          51m   v1.24.1
w2     Ready                      <none>          51m   v1.24.1

kubeadm upgrade

#檢查下載下來的 kubeadm 版本 $ kubeadm version

#將 kubeadm 升級至指定版本 : 1.24.2，可以不用打

$ sudo kubeadm upgrade apply v1.24.2 –etcd-upgrade=false

$ kubectl apply -f pod.yaml

成功後，會看到的螢幕輸出

[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.24.2". Enjoy!

[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.

Upgrade kubelet and kubectl

$ apt-mark unhold kubelet kubectl && \
  apt-get update && apt-get install -y kubelet=1.24.2-00 kubectl=1.24.2-00 && \
  apt-mark hold kubelet kubectl

Restart the kubelet

# Reload systemd manager configuration
$ systemctl daemon-reload

# 將 kubelet 重啟
$ systemctl restart kubelet

# 檢查 kubelet 狀態
$ systemctl status kubelet

回到 bigred
Uncordon the node

# Bring the node back online by marking it schedulable
$ kubectl uncordon m1
檢查 node 狀態

$ kubectl get nodes

7. Service

Reconfigure the existing deployment front-end and add a port specifiction named http exposing port 80/tcp of the existing container nginx.

Create a new service named front-end-svc exposing the container port http.

Configure the new service to also expose the individual Pods via a NodePort on the nodes on which they are scheduled.

解答

參考文章：https://blog.cptsai.com/2020/11/15/k8s-external-traffic-policy/

K8S DOC 查詢關鍵字 : nodeport yaml
找 Connecting Applications with Services | Kubernetes 這篇
service/networking/nginx-secure-app.yaml 這個 YAML 檔

匯出 YAML 檔 $ kubectl get deploy front-end -o yaml > deploy.yaml

編輯 YAML 檔

apiVersion: apps/v1
kind: Deployment
......
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - name: http                      # add
          protocol: TCP                   # add
          containerPort: 80               # add

$ k get pod –show-labels

front-end-6bc5dc8cbb-mv7hb   1/1     Running   0          33m     app=front-end,pod-template-hash=6bc5dc8cbb

重新建立

$ kubectl replace -f deploy.yaml –force

編輯 Service YAML 檔

# externalTrafficPolicy: Local   預設是 Cluster，Local 代表這個 node 要有 pod 才能透過 nodeport 連進來
apiVersion: v1
kind: Service
metadata:
  name: front-end-svc
#  labels:                                # remove
#    run: my-nginx                        # remove
spec:
  type: NodePort
  externalTrafficPolicy: Local 
# internalTrafficPolicy: Local            # 不確定，看文章
  ports:
  - port: 80
    targetPort: 80                       
    protocol: TCP
    name: http
#  - port: 443                            # remove
#    protocol: TCP                        # remove
#    name: https                          # remove
  selector:
    run: front-end                        # 看考試時的標籤

產生 Service

$ kubectl apply -f srv.yaml

用命令產生 $ kubectl expose deployment front-end –port=80 –target-port=80 –type=NodePort –name=front-end-svc

8. RBAC

Context You have been asked to create a new ClusterRole for a deployment pipeline and bind it to a specific ServiceAccount scoped a specific namespace.

Task Create a new ClusterRole named deployment-clusterrole that only allows the creation of the following resource types:

Deployment
StatefulSet
DaemonSet Create a new ServiceAccount named cicd-token in the existing namespace app-team1.

Limited to namespace app-team1, bind the new ClusterRole deployment-clusterrole to the new ServiceAccount cicd-token.

解答

建立 clusterrole $ kubectl create clusterrole deployment-clusterrole –verb=create –resource=Deployment,StatefulSet,DaemonSet
建立 ServiceAccount $ kubectl create sa cicd-token -n app-team1
建立 rolebinding $ kubectl create -n app-team1 rolebinding haha
–clusterrole=deployment-clusterrole
–serviceaccount=app-team1:cicd-token
檢查

$ k create deploy pipi -n app-team1 --image=quay.io/cloudwalker/alpine --as system:serviceaccount:app-team1:cicd-token

$ k get all -n app-team1

$ k delete deployment.apps/pipi -n app-team1 --as system:serviceaccount:app-team1:cicd-token

-Error from server (Forbidden): deployments.apps "pipi" is forbidden: User "system:serviceaccount:app-team1:cicd-token" cannot delete resource "deployments" in API group "apps" in the namespace "app-team1"

使用 can-i 檢查

$ k auth can-i --list -n app-team1 --as system:serviceaccount:app-team1:cicd-token

$ k auth can-i create pod -n app-team1 --as system:serviceaccount:app-team1:cicd-token
no

$ k auth can-i create deploy -n app-team1 --as system:serviceaccount:app-team1:cicd-token
yes

$ k auth can-i create sts -n app-team1 --as system:serviceaccount:app-team1:cicd-token
yes

$ k auth can-i create ds -n app-team1 --as system:serviceaccount:app-team1:cicd-token
yes

9. Check Ready Node

Check to see how many nodes are ready (not including nodes tained NoSchedule) and write the number to /opt/KUSC00402/kusc00402.txt.

解答

$ kubectl get nodes | grep 'Ready'

$ kubectl describe node | grep ‘NoSchedule’ $ echo “{數量}” > /opt/KUSC00402/kusc00402.txt

10. PVC

Create a new PersistentVolumeClaim:

Name: pv-volume
Class: csi-hostpath-sc
Capacity: 10Mi

Create a new Pod which mounts the PersistentVolumeClaim as a volume:

Name: web-server
Image: nginx
Mount Path: /usr/share/nginx/html

Configure the new Pod to have ReadWriteOnce access on the volume.

Finally,using kubectl edit or kubectl patch expand the PersistentVolumeClaim to a capacity of 70Mi and ==record that change.==

解答

K8S DOC 查詢關鍵字 : pod pv
找 Configure a Pod to Use a PersistentVolume for Storage | Kubernetes 這篇
找 pods/storage/pv-claim.yaml 這篇 YAML 檔
和 pods/storage/pv-pod.yaml 這篇 YAML 檔編輯 PVC YANL 檔

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pv-volume                         # change
spec:
  storageClassName: csi-hostpath-sc       # change
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi                       # change

產生 PVC

$ kubectl apply -f pvc.yaml 編輯 pod YAML

$ kubectl run web-server –image=nginx –dry-run=client -o yaml > pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: web-server                              # change
spec:
  volumes:                                      # add
    - name: task-pv-storage                     # add
      persistentVolumeClaim:                    # add
        claimName: pv-volume                    # change
  containers:
    - name: task-pv-container
      image: nginx
      ports:
        - containerPort: 80
          name: "http-server"
      volumeMounts:                             # add
        - mountPath: "/usr/share/nginx/html"    # add
          name: task-pv-storage                 # add

產生 Pod $ kubectl apply -f pod.yaml

將 PVC 的 capacity 調成 70Mi 用 nano 編輯 $ export KUBE_EDITOR=nano

修改 YAML 檔，將 capacity 設為 70Mi kubectl get pvc pv-volume

==記得加 record== $ k edit pvc pv-volume –record

#檢查
$ k get pvc pv-volume

11. Pod Logs

Monitor the logs of pod bar and:

Extract log lines corresponding to error unable-to-access-website
Write them to /opt/KUTR00101/bar

解答

$ kubectl logs bar | grep 'unable-to-access-website' > /opt/KUTR00101/bar

12. PV

題型 1: Create a persistent volume with name app-config, of capacity 1Gi and access mode ReadOnlyMany, the type of volume is hostPath and its location is /svc/app-config.

題型 2： Create a persistent volume with name app-config, of capacity 2Gi and access mode ReadWriteMany, the type of volume is hostPath and its location is /srv/app-config.

題型 1: 解答

K8S DOC 查詢關鍵字 : pod pv 找 Configure a Pod to Use a PersistentVolume for Storage | Kubernetes 這篇找 pods/storage/pv-volume.yaml 這篇 YAML 檔編輯 PV YAML 檔

apiVersion: v1
kind: PersistentVolume
metadata:
  name: app-config                      # change
#  labels:                              # remove
#    type: local                        # remove
spec:
#  storageClassName: manual             # remove
  capacity:
    storage: 1Gi                        # change
  accessModes:
    - ReadOnlyMany                      # change
  hostPath:
    path: "/svc/app-config"             # change

產生 PV

$ kubectl apply -f pv.yaml

題型 2: 解答

編輯 PV YAML 檔

apiVersion: v1
kind: PersistentVolume
metadata:
  name: app-config                      # change
#  labels:                              # remove
#    type: local                        # remove
spec:
#  storageClassName: manual             # remove
  capacity:
    storage: 2Gi                        # change
  accessModes:
    - ReadWriteMany                     # change
  hostPath:
    path: "/srv/app-config"             # change

產生 PV

$ kubectl apply -f pv.yaml

13. cordon & drain

Set the node named ek8s-node-1 as unavaliable and reschedule all pods running on it.

解答

參考連結: https://kubernetes.io/docs/concepts/workloads/pods/disruptions/

#不可以在安排 pod 在這個 node 產生 $ kubectl cordon ek8s-node-1

$ kubectl drain ek8s-node-1 –ignore-daemonsets –delete-emptydir–data –force #把 pod 趕到別的 node #–ignore-daemonsets 忽略 DaemonSet 管理的 pod #–force 把沒有 controller 管理的 pod 刪除

Applying a Node Cordon
Cordoning a Node marks it as unavailable to the Kubernetes scheduler. The Node will be ineligible to host any new Pods subsequently added to your cluster.

Use the kubectl cordon command to place a cordon around a named Node:

$ kubectl cordon node-1
node/node-1 cordoned
Existing Pods already running on the Node won’t be affected by the cordon. They’ll remain accessible and will still be hosted by the cordoned Node.

You can check which of your Nodes are currently cordoned with the get nodes command:

$ kubectl get nodes
NAME       STATUS                     ROLES                  AGE   VERSION
node-1     Ready,SchedulingDisabled   control-plane,master   26m   v1.23.3
Cordoned nodes appear with the SchedulingDisabled status.

Draining a Node
The next step is to drain remaining Pods out of the Node. This procedure will evict the Pods so they’re rescheduled onto other Nodes in your cluster. Pods are allowed to gracefully terminate before they’re forcefully removed from the target Node.

Run kubectl drain to initiate a drain procedure. Specify the name of the Node you’re taking out for maintenance:

# drain 完後要等一下子讓 pod 在別的 node 產生
$ kubectl drain node-1
node/node-1 already cordoned
evicting pod kube-system/storage-provisioner
evicting pod default/nginx-7c658794b9-zszdd
evicting pod kube-system/coredns-64897985d-dp6lx
pod/storage-provisioner evicted
pod/nginx-7c658794b9-zszdd evicted
pod/coredns-64897985d-dp6lx evicted
node/node-1 evicted
The drain procedure first cordons the Node if you’ve not already placed one manually. It will then evict running Kubernetes workloads by safely rescheduling them to other Nodes in your cluster.

You can shutdown or destroy the Node once the drain’s completed. You’ve freed the Node from its responsibilities to your cluster. The cordon provides an assurance that no new workloads have been scheduled since the drain completed.

檢查

$ kubectl get node

14. Trobleshooting - kubelet 故障

A Kubernetes worker node, named wk8s-node-0 is in state NotReady. Investigate why this is the case, and perform an appropriate steps to bring the node to a Ready stat, ensuring that any changes are made permanent.

You can ssh to the failed node using: ssh wk8s-node-0 You can assume elevated privileges on the node with the following command: sudo -i

解答

#檢查
$ kubectl get nodes

# 連到 wk8s-node-0 這台 node
$ ssh wk8s-node-0

# 提權
$ sudo -i

# 檢查 kubelet 狀態
$ systemctl status kubelet

# 將 kubelet 重啟
$ systemctl start kubelet

# 將 kubelet 設為開機自動啟動
$ systemctl enable kubelet

# 檢查 kubelet 狀態
$ systemctl status kubelet

15. Sidecar

Context Without changing its existing containers, an existing Pod needs to be integrated into Kubernetes’s build-in logging architecture(e.g kubectl logs). Adding a streaming sidecar container is a good and common way to accomplish this requirement.

Task Add a busybox sidecar container to the existing Pod big-corp-app.The new sidecar container has to run the following command:

/bin/sh -c tail -n+1 -f /var/log/big-corp-app.log Use a volume mount named logs to make the file /var/log/big-corp-app.log available to the sidecar container.

Don’t modify the existing container. Don’t modify the path of the log file, both containers must access it at /var/log/bin-corp-app.log

解答

原本的yaml 檔，已經存在

apiVersion: v1
kind: Pod
metadata:
  name: big-corp-app
spec:
  containers:
  - name: big-corp-app
    image: busybox
    args:
    - /bin/sh
    - -c
    - |
      mkdir -p /var/log;
      i=0;
      while true;
      do
        echo "$(date) INFO $i" >> /var/log/big-corp-app.log;
        i=$((i+1));
        sleep 1;
      done

測試： 看 log 裡面沒有任何東西 $ k logs big-corp-app

$ tail -n+1 -f aaa 
這行命令是 tail 會一直去看 aaa 這個檔案
# -f (file)就是會一直去看檔案

範例
$ tail -n+1 -f aaa
eee
123
haha
yyy

開啟另一個終端機
$ echo eee >> aaa
$ echo 123 >> aaa
$ echo haha >> aaa
$ echo yyy >> aaa

再建一個 Container，名字叫 count-log，要去收集已經存在的 Container log 資訊，答案如下：

答案： K8S DOC 查詢關鍵字 : sidecar yaml 找 Logging Architecture | Kubernetes 這篇 ctrl + f 搜尋 : streaming- 匯出 YAML 檔

$ k get pod big-corp-app -o yaml > sidecar.yaml 編輯 YAML 檔，要刪除原本的 volumes

apiVersion: v1
kind: Pod
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"name":"big-corp-app","namespace":"default"},"spec":{"containers":[{"args":["/bin/sh","-c","mkdir -p /var/log;\ni=0;\nwhile true;\ndo\n  echo \"$(date) INFO $i\" \u003e\u003e /var/log/big-corp-app.log;\n  i=$((i+1));\n  sleep 1;\ndone\n"],"image":"quay.io/cloudwalker/busybox","name":"big-corp-app"}]}}
  creationTimestamp: "2022-09-29T14:47:24Z"
  name: big-corp-app
  namespace: default
  resourceVersion: "34289"
  uid: b8598b00-8c2c-4014-80f6-84fa7af04712
spec:
  containers:
  - args:
    - /bin/sh
    - -c
    - |
      mkdir -p /var/log;
      i=0;
      while true;
      do
        echo "$(date) INFO $i" >> /var/log/big-corp-app.log;
        i=$((i+1));
        sleep 1;
      done
    image: quay.io/cloudwalker/busybox                               
    imagePullPolicy: Always                                          
    name: big-corp-app       
    resources: {}
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File                                     
    volumeMounts:                                                  
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount      
      name: kube-api-access-w2w4r                                   
      readOnly: true                                               
    volumeMounts:                                                    # add
    - name: logs                                                     # add
      mountPath: /var/log                                            # add
  - name: count-log                                                  # add
    image: busybox                                                   # add
    args: [/bin/sh, -c, 'tail -n+1 -f /var/log/big-corp-app.log']    # add
    volumeMounts:                                                    # add
    - name: logs                                                     # add
      mountPath: /var/log                                            # add
  volumes:                                                           # add
  - name: logs                                                       # add
    emptyDir: {}                                                     # add
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  nodeName: w1
  preemptionPolicy: PreemptLowerPriority
  priority: 0
  restartPolicy: Always
  schedulerName: default-scheduler
  securityContext: {}
  serviceAccount: default
  serviceAccountName: default
  terminationGracePeriodSeconds: 30
  tolerations:
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  volumes:                                                          
  - name: kube-api-access-w2w4r                                     
    projected:                                                     
      defaultMode: 420                                             
      sources:                                                     
      - serviceAccountToken:                                       
          expirationSeconds: 3607                                  
          path: token                                              
      - configMap:                                                 
          items:                                                    
          - key: ca.crt                                             
            path: ca.crt                                            
          name: kube-root-ca.crt                                    
      - downwardAPI:                                               
          items:                                                    
          - fieldRef:                                               
              apiVersion: v1                                        
              fieldPath: metadata.namespace                         
            path: namespace                                         
status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: "2022-09-29T14:47:24Z"
    status: "True"
    type: Initialized
  - lastProbeTime: null
    lastTransitionTime: "2022-09-29T14:47:28Z"
    status: "True"
    type: Ready
  - lastProbeTime: null
    lastTransitionTime: "2022-09-29T14:47:28Z"
    status: "True"
    type: ContainersReady
  - lastProbeTime: null
    lastTransitionTime: "2022-09-29T14:47:24Z"
    status: "True"
    type: PodScheduled
  containerStatuses:
  - containerID: cri-o://a6a13dcf0b11f0e6fc22eb507b23cfde7360995d24af798e7630ba26ddebb6d1
    image: quay.io/cloudwalker/busybox:latest
    imageID: quay.io/cloudwalker/busybox@sha256:f3cfc9d0dbf931d3db4685ec659b7ac68e2a578219da4aae65427886e649b06b
    lastState: {}
    name: big-corp-app
    ready: true
    restartCount: 0
    started: true
    state:
      running:
        startedAt: "2022-09-29T14:47:27Z"
  hostIP: 192.168.61.241
  phase: Running
  podIP: 10.244.1.16
  podIPs:
  - ip: 10.244.1.16
  qosClass: BestEffort
  startTime: "2022-09-29T14:47:24Z"

產生pod $ kubectl replace -f 15.yaml –force

$ kubectl get pod big-corp-app 檢查 $ kubectl logs big-corp-app -c count-log

16.1 Ingress

Create a new nginx Ingress resources as follows:

Name: ping
Namespace: ing-internal
Exposing service hi on path /hi using service port 5678

The avaliability of service hi can be checked using the following command,which should return hi: curl -kL <INTERNAL_IP>/hi

環境準備

安裝 metallb

$ wget -qO - https://raw.githubusercontent.com/metallb/metallb/v0.13.4/config/manifests/metallb-native.yaml | kubectl apply -f -

echo '
 apiVersion: metallb.io/v1beta1
 kind: IPAddressPool
 metadata:
   name: mlb1
   namespace: metallb-system
 spec:
   addresses:
     - 120.96.143.30-120.96.143.50
 ---
 apiVersion: metallb.io/v1beta1
 kind: L2Advertisement
 metadata:
   name: mlb1
   namespace: metallb-system' | kubectl apply -f -

$ kubectl get pod -n metallb-system -o wide

$ kubectl get service -n ingress-nginx

$ kubectl get ingressclass

$ k run n1 --image=nginx -n ing-internal -l run=nginx

$ kg all -n ing-internal

安裝 ingress $ wget -qO - https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.3.0/deploy/static/provider/cloud/deploy.yaml | sed 's|name: nginx|name: ig1|g' | kubectl apply -f -

解答

K8S DOC 查詢關鍵字 : ingress
找 Ingress | Kubernetes 這篇
用第一個 YANL 檔編輯 YAML 檔

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ping
  annotations: 
    kubernetes.io/ingress.class: "nginx"           # 加這行 ingress 就會有 address 
    nginx.ingress.kubernetes.io/rewrite-target: /
  namespace: ing-internal
spec:
  rules:
  - http:
      paths:
      - path: /hi
        pathType: Prefix
        backend:
          service:
            name: hi
            port:
              number: 5678

產生 ingress

$ k apply -f ingress.yaml 測試

$ curl -kL <INTERNAL_IP>/hi

$ curl -w '\n' --resolve k8s.org:80:120.96.143.30 http://k8s.org/hi

16.2 Kubernetes 1.25.0 版的 Ingress 範例操作

$ kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.13.5/config/manifests/metallb-native.yaml

$ echo '
 apiVersion: metallb.io/v1beta1
 kind: IPAddressPool
 metadata:
   name: mlb1
   namespace: metallb-system
 spec:
   addresses:
     - 192.168.61.220-192.168.61.230' | kubectl apply -f -

$ curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash

$ helm repo add nginx-stable https://helm.nginx.com/stable; helm repo update

$ helm search repo nginx-stable

$ helm install ig1 nginx-stable/nginx-ingress --set controller.ingressClass="ig1"

$ wget -qO - http://www.oc99.org/kube/apple-banana.yaml | kubectl apply -f -

$ nano ig1.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ig1
  annotations:
    kubernetes.io/ingress.class: ig1
spec:
  rules:
  - host: ig1.k8s.org
    http:
      paths:
      - path: /apple
        pathType: Prefix
        backend:
          service:
            name: apple-service
            port:
              number: 5678
      - path: /banana
        pathType: Prefix
        backend:
          service:
            name: banana-service
            port:
              number: 5678

$ kubectl apply -f ig1.yaml

$ kubectl get ingress
NAME   CLASS    HOSTS         ADDRESS          PORTS   AGE
ig1    <none>   ig1.k8s.org   120.96.143.240   80      25m

$ curl -k -H "Host: ig1.k8s.org" http://120.96.143.240/apple/
apple

17. Etcd （7%）

First, create a snapshot of the existing etcd instance running at https://127.0.0.1:2379, saving the snapshot to /var/lib/backup/etcd-snapshot.db

Creating a snapshot of the given instance is expected to complete in seconds. If the operation seems to hang, something’s likely wrong with your command. Use CTRL+C to cancel the operation and try again.

Next, restore an existing, previous snapshot located at /var/lib/backup/etcd-snapshot-previous.db.

The following TLS certificates/key are supplied for connecting to the server with etcdctl: CA certificate: /opt/KUIN0061/ca.crt Client certificate: /opt/KUIN0061/etcd-client.crt Client key: /opt/KUIN0061/etcd-client.key

環境準備

ubuntu 安裝 etcdctl 命令 $ sudo apt install etcd-client

alpine 安裝 etcdctl 命令

$ sudo apk add etcd-ctl \
  --no-cache \
  --update-cache \
  --allow-untrusted \
  --repository http://dl-cdn.alpinelinux.org/alpine/edge/testing

解答

K8S DOC 查詢關鍵字 : etcd backup
點選 Operating etcd clusters for Kubernetes | Kubernetes 這篇文章
進取之後點藍藍的 backup
將 etcd 備份 (考試環境，確認 master node 有無 etcdctl 命令，如沒有則在一開始的終端機做題 )
參考文章 : https://github.com/bmuschko/cka-crash-course/blob/master/exercises/04-etcd-backup-restore/solution/solution.md

# 如果題目沒有給檔案路徑就要到 etcd.yaml 這個檔案去找
$ sudo cat /etc/kubernetes/manifests/etcd.yaml | grep file
    - --cert-file=/etc/kubernetes/pki/etcd/server.crt # 這行
    - --key-file=/etc/kubernetes/pki/etcd/server.key # 這行
    - --peer-cert-file=/etc/kubernetes/pki/etcd/peer.crt
    - --peer-key-file=/etc/kubernetes/pki/etcd/peer.key
    - --peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt
    - --trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt # 這行
    seccompProfile:

使用題目給的檔案路徑

$ ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 \
  --cacert=/opt/KUIN0061/ca.crt \
  --cert=/opt/KUIN0061/etcd-client.crt \
  --key=/opt/KUIN0061/etcd-client.key \
  snapshot save /var/lib/backup/etcd-snapshot.db

檢查 save 有沒有成功可以下以下這行指令

$ sudo ETCDCTL_API=3 etcdctl --write-out=table snapshot status /var/lib/backup/etcd-snapshot.db

將 etcd 還原 save、restore 在本機操作

$ ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 \
  --data-dir /var/lib/etcd-1 \      # 要加 data-dir 這個參數
  --cacert=/opt/KUIN0061/ca.crt \
  --cert=/opt/KUIN0061/etcd-client.crt \
  --key=/opt/KUIN0061/etcd-client.key \
  snapshot restore /var/lib/backup/etcd-snapshot-previous.db

修改 yaml 檔要 ssh 進去 master 機

# 修改 etcd.yaml 檔案
$ sudo nano /etc/kubernetes/manifests/etcd.yaml
........
    volumeMounts:
    - mountPath: /var/lib/etcd
      name: etcd-data
    - mountPath: /etc/kubernetes/pki/etcd
      name: etcd-certs
  hostNetwork: true
  priorityClassName: system-node-critical
  securityContext:
    seccompProfile:
      type: RuntimeDefault
  volumes:
  - hostPath:
      path: /etc/kubernetes/pki/etcd
      type: DirectoryOrCreate
    name: etcd-certs
  - hostPath:
      path: /var/lib/etcd-1    # 修改路徑
      type: DirectoryOrCreate
    name: etcd-data
status: {}

檢查

$ kubectl get pod

alpine etcd 備份

$ ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 \
  --cacert=/opt/KUIN0061/ca.crt \
  --cert=/opt/KUIN0061/etcd-client.crt \
  --key=/opt/KUIN0061/etcd-client.key \
  snapshot save /var/lib/backup/etcd-snapshot.db

將 etcd 還原

$ ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 \
  --data-dir /var/lib/etcd-1 \      # 要加 data-dir 這個參數
  --cacert=/opt/KUIN0061/ca.crt \
  --cert=/opt/KUIN0061/etcd-client.crt \
  --key=/opt/KUIN0061/etcd-client.key \
  snapshot restore /var/lib/backup/etcd-snapshot.db

# 修改 etcd.yaml 檔案
$ sudo nano /etc/kubernetes/manifests/etcd.yaml
........
    volumeMounts:
    - mountPath: /var/lib/etcd
      name: etcd-data
    - mountPath: /etc/kubernetes/pki/etcd
      name: etcd-certs
  hostNetwork: true
  priorityClassName: system-node-critical
  securityContext:
    seccompProfile:
      type: RuntimeDefault
  volumes:
  - hostPath:
      path: /etc/kubernetes/pki/etcd
      type: DirectoryOrCreate
    name: etcd-certs
  - hostPath:
      path: /var/lib/etcd-1    # 修改路徑
      type: DirectoryOrCreate
    name: etcd-data
status: {}

檢查

$ kubectl get pod

tags: `CKA`

J509教室windows電腦設定 Build HA K8S - 3M

CKA

CKA

考試環境圖

1. Scale

環境準備

解答

2. NetworkPolicy （7%）

解答

驗證

2.2 NetworkPolicy

解答

3. Multi-container

解答

4. CPU

環境準備

解答

5. NodeSelector

解答

6. Kubernetes 升級

7. Service

解答

8. RBAC

解答

9. Check Ready Node

解答

10. PVC

解答

11. Pod Logs

解答

12. PV

題型 1: 解答

題型 2: 解答

13. cordon & drain

解答

檢查

14. Trobleshooting - kubelet 故障

解答

15. Sidecar

解答

16.1 Ingress

環境準備

解答

16.2 Kubernetes 1.25.0 版的 Ingress 範例操作

17. Etcd （7%）

環境準備

解答

alpine etcd 備份

tags: CKA

tags: `CKA`