antony@notes:~/rancher$ cat "RKE2-Rancher-Prime-Installation.md"

RKE2 & Rancher Prime Installation

2026-03-31· Rancher / RKE2 ·SUSE Rancher 2.6

Preface

本篇文章會主要會介紹，如何安裝 Rancher Kubernetes Engine 2 或 K3S，並在 Kubernetes 之中部署 Rancher Prime

可以透過點擊以下目錄，選擇想看的內容，跳轉至特定章節

Install RKE2 v1.34.4

一、安裝前環境檢查

請確保你的環境符合要求
- RKE2 Requirements
- 如果你的主機上安裝並啟用了 NetworkManager，確保它被配置為忽略 CNI 管理的接口。
對於 1.21 及以上版本的 RKE2，如果主機核心支持 AppArmor，那麼在安裝 RKE2 之前還必須有 AppArmor 工具（通常通過 apparmor-parser 軟件包獲得）。
RKE2 的安裝過程必須以 root 用戶或通過 sudo 運行。

Hardware

Linux/Windows

RAM: 4GB Minimum (we recommend at least 8GB)
CPU: 2 Minimum (we recommend at least 4CPU)
Disks: SSD

RKE2 Hardware

二、Server Node Installation

1. Config RKE2 Basic Parameters

1-1. 下載 RKE2 的安裝腳本，並賦予執行權限

在 Server node 的機器上執行以下命令

curl -sfL https://get.rke2.io --output install.sh && chmod +x install.sh

RKE2 提供了一個安裝腳本，在 systemd 的系統上將其作為一個服務來安裝。

1-2. 編輯 Server Node 設定檔

advertise-address, node-ip 和 tls-san 的 IP 需要依據不同的環境來設定。

# 定義 K8s 要跑在哪張網卡上(以下指令預設會去找 default gateway 的網卡)
NIC="$(ip route show default | cut -d' ' -f5)"
IP=$(ip -4 -o addr show ${NIC} | awk '{print $4}' | cut -d/ -f1)

# 定義節點名稱
NODE_NAME="$(hostname -f 2>/dev/null)"
NODE_NAME="${NODE_NAME:-$(hostname)}"

sudo mkdir -p /etc/rancher/rke2/ && \
cat <<EOF | sudo tee /etc/rancher/rke2/config.yaml
write-kubeconfig-mode: "0644"
debug: false

# =============================================
# 每台節點不同的設定
# =============================================

# 本機對外廣播的 IP（其他節點透過此 IP 連線）
advertise-address: "${IP}"
# 本機 IP
node-ip: "${IP}"
# 節點名稱（每台不同）
node-name: "${NODE_NAME}"

# TLS 憑證允許的 SAN（第一台 server 需完整設定）
tls-san:
  - "${NODE_NAME}"
  - "${IP}"
  - "127.0.0.1"

# 第二台以後需加入此行，指向第一台 server（第一台不需要）
# server: "https://<第一台IP>:9345"

# =============================================
# 所有 Server Node 必須相同的設定
# =============================================

# 叢集加入金鑰
token: "hehe"

# 各元件時區設定
etcd-extra-env:
  - "TZ=Asia/Taipei"
kube-apiserver-extra-env:
  - "TZ=Asia/Taipei"
kube-controller-manager-extra-env:
  - "TZ=Asia/Taipei"
kube-scheduler-extra-env:
  - "TZ=Asia/Taipei"
cloud-controller-manager-extra-env:
  - "TZ=Asia/Taipei"

# kubelet 參數
kubelet-arg:
  - "container-log-max-files=5"           # 最多保留 5 個 log 檔
  - "container-log-max-size=10Mi"         # 每個 log 檔最大 10Mi
  - "config=/etc/kubernetes/kubeletconfig.yml"
  - "eviction-hard=memory.available<1000Mi,imagefs.available<5%,nodefs.available<5%"  # 資源不足時驅逐 Pod 的閾值

# 停用內建 cloud controller
disable-cloud-controller: true
# 停用 kube-proxy，改由 Cilium 接管
disable-kube-proxy: true

# 停用不需要的內建元件
disable:
  - rke2-canal       # 停用預設 CNI Canal
  - rke2-kube-proxy  # 停用內建 kube-proxy

# 使用 Cilium 作為 CNI
cni:
  - "cilium"

# 啟用 CIS security profile
#profile: "cis"

# API Server 額外參數
#kube-apiserver-arg:
#  - "service-account-extend-token-expiration=false"

EOF

1-3. 編輯 cilium 設定檔

sudo mkdir -p /var/lib/rancher/rke2/server/manifests && \
cat <<EOF | sudo tee /var/lib/rancher/rke2/server/manifests/rke2-cilium-config.yaml
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: rke2-cilium
  namespace: kube-system
spec:
  valuesContent: |-
    operator:
      replicas: 1
    podAnnotations:
      container.apparmor.security.beta.kubernetes.io/cilium-agent: "unconfined"
      container.apparmor.security.beta.kubernetes.io/clean-cilium-state: "unconfined"
      container.apparmor.security.beta.kubernetes.io/mount-cgroup: "unconfined"
      container.apparmor.security.beta.kubernetes.io/apply-sysctl-overwrites: "unconfined"
    devices: "$NIC"
    nodePort:
      directRoutingDevice: "$NIC"
    autoDirectNodeRoutes: true
    hubble:
      enabled: false
      relay:
        enabled: false
      ui:
        enabled: false
        frontend:
          server:
            ipv6:
              enabled: false
      service:
        type: NodePort
    enableLBIPAM: false
    ipv4NativeRoutingCIDR: 10.42.0.0/16
    k8sServiceHost: 127.0.0.1
    k8sServicePort: 6443
    kubeProxyReplacement: true
    routingMode: native
    l2announcements:
      enabled: true
EOF

1-4. 編輯 kubelet 設定檔

sudo mkdir -p /etc/kubernetes && \
cat <<EOF | sudo tee /etc/kubernetes/kubeletconfig.yml
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
systemReserved:
  memory: "1Gi"
kubeReserved:
  memory: "2Gi"
EOF

evictionHard：
- 用途：設定強制驅逐的值，當系統資源（如記憶體）低於此設定值時，kubelet 會立即終止部分 Pod，以釋放資源，確保系統穩定運行。
- 預設值: 預設打開，Memory 是 100MiB, Ephemeral-Storage 是 10% (開源的 K8s), rke2 預設值是 imagefs 和 nodefs = 5%
- 範例：memory.available: "500Mi" 表示當可用記憶體少於 500MiB 時，kubelet 會觸發驅逐機制。
evictionMinimumReclaim：
- 用途：設定在驅逐 Pod 時，kubelet 希望至少回收的資源量。這有助於避免頻繁的驅逐操作，確保每次驅逐能有效釋放足夠的資源。
- 範例：memory.available: "10Mi" 表示每次驅逐操作至少要回收 10MiB 的記憶體。
systemReserved：
- 用途：為系統背景程序（如 sshd、networking 等非 Kubernetes 組件）預留資源，確保這些系統服務在資源緊張時仍能正常運行。
- 範例：memory: "1Gi" 表示為系統服務預留 1GiB 的記憶體。
kubeReserved：
- 用途：為 Kubernetes 系統的背景程序（如 kubelet、container runtime 時等）預留資源，確保這些組件在資源緊張時的穩定性。
- 範例：memory: "2Gi" 表示為 Kubernetes 系統的背景程序預留 2GiB 的記憶體。

2. Install RKE2 with v1.34.4 version

2-1. 執行以下安裝命令

sudo INSTALL_RKE2_CHANNEL=v1.34.4+rke2r1 ./install.sh

INSTALL_RKE2_CHANNEL
指定要下載 RKE2 URL 的 Channel，預設是 latest

[INFO]  finding release for channel v1.34.4+rke2r1
[INFO]  using v1.34.4+rke2r1 as release
[INFO]  downloading checksums at https://github.com/rancher/rke2/releases/download/v1.34.4%2Brke2r1/sha256sum-amd64.txt
[INFO]  downloading tarball at https://github.com/rancher/rke2/releases/download/v1.34.4%2Brke2r1/rke2.linux-amd64.tar.gz
[INFO]  verifying tarball
[INFO]  unpacking tarball file to /usr/local

3. enable rke2 and setup kubeconfig

3-1. 將 rke2-server 的服務設為開機自動啟動，並立即啟動服務

sudo systemctl enable rke2-server --now

以上命令執行後的螢幕輸出

Created symlink /etc/systemd/system/multi-user.target.wants/rke2-server.service → /etc/systemd/system/rke2-server.service.

3-2. Setup kubeconfig and crictl

{
  # 1. 建立目錄並設定 Kubeconfig
  mkdir -p $HOME/.kube && \
  sudo cp /etc/rancher/rke2/rke2.yaml $HOME/.kube/config && \
  sudo chown $(id -u):$(id -g) $HOME/.kube/config && \
  
  # 2. 搬移執行檔
  sudo cp /var/lib/rancher/rke2/bin/* /usr/local/bin/ && \
  
  # 3. 寫入 crictl 設定 (使用 printf 保持單行邏輯)
  printf "runtime-endpoint: unix:///run/k3s/containerd/containerd.sock\nimage-endpoint: unix:///run/k3s/containerd/containerd.sock\ntimeout: 10\n" | sudo tee /etc/crictl.yaml > /dev/null && \
  
  # 4. 設定 Bash Alias 與補全
  printf "\nsource <(kubectl completion bash)\nalias k=kubectl\ncomplete -o default -F __start_kubectl k\n" >> ~/.bashrc && \
  
  # 5. 立即生效
  source ~/.bashrc
}

4. check pod status

kubectl get po -A

螢幕輸出 :

NAMESPACE     NAME                                                    READY   STATUS      RESTARTS   AGE
kube-system   cilium-j7jf2                                            1/1     Running     0          2m12s
kube-system   cilium-operator-6fb6d5b896-jj2nx                        1/1     Running     0          2m12s
kube-system   etcd-rms-1.kubeantony.com                               1/1     Running     0          2m31s
kube-system   helm-install-rke2-cilium-qhdsl                          0/1     Completed   0          2m24s
kube-system   helm-install-rke2-coredns-hwsqp                         0/1     Completed   0          2m24s
kube-system   helm-install-rke2-ingress-nginx-bxt7n                   0/1     Completed   0          2m24s
kube-system   helm-install-rke2-metrics-server-qln6s                  0/1     Completed   0          2m24s
kube-system   helm-install-rke2-runtimeclasses-5kj6v                  0/1     Completed   0          2m23s
kube-system   helm-install-rke2-snapshot-controller-crd-vsbjv         0/1     Completed   0          2m23s
kube-system   helm-install-rke2-snapshot-controller-z9kfv             0/1     Completed   1          2m23s
kube-system   kube-apiserver-rms-1.kubeantony.com                     1/1     Running     0          2m31s
kube-system   kube-controller-manager-rms-1.kubeantony.com            1/1     Running     0          2m31s
kube-system   kube-scheduler-rms-1.kubeantony.com                     1/1     Running     0          2m31s
kube-system   rke2-coredns-rke2-coredns-77f54456d4-kqbtc              1/1     Running     0          2m12s
kube-system   rke2-coredns-rke2-coredns-autoscaler-6c58bf4677-wj7gp   1/1     Running     0          2m12s
kube-system   rke2-ingress-nginx-controller-bhf7h                     1/1     Running     0          62s
kube-system   rke2-metrics-server-69f4f95bf7-tb6rc                    1/1     Running     0          76s
kube-system   rke2-snapshot-controller-bc4c494bf-cqj7t                1/1     Running     0          76s

三、Join Additional Server Nodes

1. Config RKE2 Basic Parameters

1-1. 下載 RKE2 的安裝腳本，並賦予執行權限

以下命令請在 Additional Server node 上執行

curl -sfL https://get.rke2.io --output install.sh && chmod +x install.sh

RKE2 提供了一個安裝腳本，在 systemd 的系統上將其作為一個服務來安裝。

1-2. 編輯 RKE2 Additional Server Node 設定檔

# 定義 K8s 要跑在哪張網卡上(以下指令預設會去找 default gateway 的網卡)
NIC="$(ip route show default | cut -d' ' -f5)"
IP=$(ip -4 -o addr show ${NIC} | awk '{print $4}' | cut -d/ -f1)

# 定義節點名稱
NODE_NAME="$(hostname -f 2>/dev/null)"
NODE_NAME="${NODE_NAME:-$(hostname)}"

# 定義第一台 Control-plane node 的 ip
ServerIP="192.168.11.161"

sudo mkdir -p /etc/rancher/rke2/ && \
cat <<EOF | sudo tee /etc/rancher/rke2/config.yaml
write-kubeconfig-mode: "0644"
debug: false

# =============================================
# 每台節點不同的設定
# =============================================

# 本機對外廣播的 IP（其他節點透過此 IP 連線）
advertise-address: "${IP}"
# 本機 IP
node-ip: "${IP}"
# 節點名稱（每台不同）
node-name: "${NODE_NAME}"

# TLS 憑證允許的 SAN（第一台 server 需完整設定）
tls-san:
  - "${NODE_NAME}"
  - "${IP}"
  - "127.0.0.1"

# 第二台以後需加入此行，指向第一台 server（第一台不需要）
server: "https://${ServerIP}:9345"

# =============================================
# 所有 Server Node 必須相同的設定
# =============================================

# 叢集加入金鑰
token: "hehe"

# 各元件時區設定
etcd-extra-env:
  - "TZ=Asia/Taipei"
kube-apiserver-extra-env:
  - "TZ=Asia/Taipei"
kube-controller-manager-extra-env:
  - "TZ=Asia/Taipei"
kube-scheduler-extra-env:
  - "TZ=Asia/Taipei"
cloud-controller-manager-extra-env:
  - "TZ=Asia/Taipei"

# kubelet 參數
kubelet-arg:
  - "container-log-max-files=5"           # 最多保留 5 個 log 檔
  - "container-log-max-size=10Mi"         # 每個 log 檔最大 10Mi
  - "config=/etc/kubernetes/kubeletconfig.yml"
  - "eviction-hard=memory.available<1000Mi,imagefs.available<5%,nodefs.available<5%"  # 資源不足時驅逐 Pod 的閾值

# 停用內建 cloud controller
disable-cloud-controller: true
# 停用 kube-proxy，改由 Cilium 接管
disable-kube-proxy: true

# 停用不需要的內建元件
disable:
  - rke2-canal       # 停用預設 CNI Canal
  - rke2-kube-proxy  # 停用內建 kube-proxy

# 使用 Cilium 作為 CNI
cni:
  - "cilium"

# 啟用 CIS security profile
#profile: "cis"

# API Server 額外參數
#kube-apiserver-arg:
#  - "service-account-extend-token-expiration=false"

EOF

Note: 要指定第一台 RKE2 Server node 的 IP

1-3. 編輯 kubelet 設定檔

sudo mkdir -p /etc/kubernetes && \
cat <<EOF | sudo tee /etc/kubernetes/kubeletconfig.yml
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
systemReserved:
  memory: "1Gi"
kubeReserved:
  memory: "2Gi"
EOF

2. Install RKE2 with v1.34.4 veersion

2-1. 執行以下安裝命令

sudo INSTALL_RKE2_CHANNEL=v1.34.4+rke2r1 ./install.sh

INSTALL_RKE2_CHANNEL
指定要下載 RKE2 URL 的 Channel，預設是 latest

[WARN]  /usr/local is read-only or a mount point; installing to /opt/rke2
[INFO]  finding release for channel v1.23.9+rke2r1
[INFO]  using v1.23.9+rke2r1 as release
[INFO]  downloading checksums at https://github.com/rancher/rke2/releases/download/v1.23.9+rke2r1/sha256sum-amd64.txt
[INFO]  downloading tarball at https://github.com/rancher/rke2/releases/download/v1.23.9+rke2r1/rke2.linux-amd64.tar.gz
[INFO]  verifying tarball
[INFO]  unpacking tarball file to /opt/rke2
[INFO]  updating tarball contents to reflect install path
[INFO]  moving systemd units to /etc/systemd/system
[INFO]  install complete; you may want to run:  export PATH=$PATH:/opt/rke2/bin

3. enable rke2 and setup kubeconfig

3-1. 將 rke2-server 的服務設為開機自動啟動，並立即啟動服務

sudo systemctl enable rke2-server --now

以上命令執行後的螢幕輸出

Created symlink /etc/systemd/system/multi-user.target.wants/rke2-server.service → /etc/systemd/system/rke2-server.service.

4. check pod status

kubectl get po -A

螢幕輸出 :

NAMESPACE     NAME                                                    READY   STATUS      RESTARTS   AGE
kube-system   cilium-79sx5                                            1/1     Running     0          2m35s
kube-system   cilium-j7jf2                                            1/1     Running     0          19m
kube-system   cilium-operator-6fb6d5b896-jj2nx                        1/1     Running     0          19m
kube-system   etcd-rms-1.kubeantony.com                               1/1     Running     0          19m
kube-system   etcd-rms-2.kubeantony.com                               1/1     Running     0          2m35s
kube-system   helm-install-rke2-cilium-qhdsl                          0/1     Completed   0          19m
kube-system   helm-install-rke2-coredns-hwsqp                         0/1     Completed   0          19m
kube-system   helm-install-rke2-ingress-nginx-bxt7n                   0/1     Completed   0          19m
kube-system   helm-install-rke2-metrics-server-qln6s                  0/1     Completed   0          19m
kube-system   helm-install-rke2-runtimeclasses-5kj6v                  0/1     Completed   0          19m
kube-system   helm-install-rke2-snapshot-controller-crd-vsbjv         0/1     Completed   0          19m
kube-system   helm-install-rke2-snapshot-controller-z9kfv             0/1     Completed   1          19m
kube-system   kube-apiserver-rms-1.kubeantony.com                     1/1     Running     0          19m
kube-system   kube-apiserver-rms-2.kubeantony.com                     1/1     Running     0          2m35s
kube-system   kube-controller-manager-rms-1.kubeantony.com            1/1     Running     0          19m
kube-system   kube-controller-manager-rms-2.kubeantony.com            1/1     Running     0          2m34s
kube-system   kube-scheduler-rms-1.kubeantony.com                     1/1     Running     0          19m
kube-system   kube-scheduler-rms-2.kubeantony.com                     1/1     Running     0          2m34s
kube-system   rke2-coredns-rke2-coredns-77f54456d4-4n5k5              1/1     Running     0          82s
kube-system   rke2-coredns-rke2-coredns-77f54456d4-kqbtc              1/1     Running     0          19m
kube-system   rke2-coredns-rke2-coredns-autoscaler-6c58bf4677-wj7gp   1/1     Running     0          19m
kube-system   rke2-ingress-nginx-controller-bhf7h                     1/1     Running     0          18m
kube-system   rke2-ingress-nginx-controller-vhmtb                     1/1     Running     0          85s
kube-system   rke2-metrics-server-69f4f95bf7-tb6rc                    1/1     Running     0          18m
kube-system   rke2-snapshot-controller-bc4c494bf-cqj7t                1/1     Running     0          18m

四、Join Agent Nodes

1. Config RKE2 Basic Parameters

1-1. 下載 RKE2 的安裝腳本，並賦予執行權限

以下命令請在 agent node 上執行

~> curl -sfL https://get.rke2.io --output install.sh && chmod +x install.sh

RKE2 提供了一個安裝腳本，在 systemd 的系統上將其作為一個服務來安裝。

1-2. 編輯 RKE2 Additional Server Node 設定檔

# 定義 K8s 要跑在哪張網卡上
$ NIC='eth1'
$ IP=$(ip -4 -o addr show ${NIC} | awk '{print $4}' | cut -d/ -f1)
# 定義 API Server 的 IP 位址
$ ServerIP='192.168.11.35'
# 定義節點名稱
$ NODE_NAME='w1'

~> sudo mkdir -p /etc/rancher/rke2/ && \
cat <<EOF | sudo tee /etc/rancher/rke2/config.yaml
server: https://${ServerIP}:9345
write-kubeconfig-mode: "0644"
debug: False
node-ip: "${IP}"
node-name:
  - "${NODE_NAME}"
token: "hehe"
etcd-extra-env: TZ=Asia/Taipei
kube-apiserver-extra-env: TZ=Asia/Taipei
kube-controller-manager-extra-env: TZ=Asia/Taipei
kube-proxy-extra-env: TZ=Asia/Taipei
kube-scheduler-extra-env: TZ=Asia/Taipei
cloud-controller-manager-extra-env: TZ=Asia/Taipei
kubelet-arg:
  - container-log-max-files=5
  - container-log-max-size=10Mi
  - "--config=/etc/kubernetes/kubeletconfig.yml"
  - "--eviction-hard=memory.available<1000Mi,imagefs.available<5%,nodefs.available<5%"
#selinux: false
EOF

Note: 要指定第一台 RKE2 Server node 的 IP

1-3. 編輯 kubelet 設定檔

~> sudo mkdir -p /etc/kubernetes && \
cat <<EOF | sudo tee /etc/kubernetes/kubeletconfig.yml
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
systemReserved:
  memory: "1Gi"
kubeReserved:
  memory: "2Gi"
EOF

2. Install RKE2 with 1.31.2 veersion

2-1. 執行以下安裝命令

~> sudo INSTALL_RKE2_TYPE="agent" INSTALL_RKE2_CHANNEL=v1.31.2+rke2r1 ./install.sh

INSTALL_RKE2_CHANNEL
指定要下載 RKE2 URL 的 Channel，預設是 latest

[INFO]  finding release for channel v1.34.4+rke2r1
[INFO]  using v1.34.4+rke2r1 as release
[INFO]  downloading checksums at https://github.com/rancher/rke2/releases/download/v1.34.4%2Brke2r1/sha256sum-amd64.txt
[INFO]  downloading tarball at https://github.com/rancher/rke2/releases/download/v1.34.4%2Brke2r1/rke2.linux-amd64.tar.gz
[INFO]  verifying tarball
[INFO]  unpacking tarball file to /usr/local

3. enable rke2

3-1. 將 rke2-agent 的服務設為開機自動啟動，並立即啟動服務

~> sudo systemctl enable rke2-agent --now

以上命令執行後的螢幕輸出

Created symlink /etc/systemd/system/multi-user.target.wants/rke2-server.service → /etc/systemd/system/rke2-server.service.

4. check pod status

以下命令在第一台 RKE2 Server 執行

~> kubectl get nodes,po -A

螢幕輸出 :

NAME      STATUS   ROLES                       AGE     VERSION
node/m1   Ready    control-plane,etcd,master   4h32m   v1.31.1+rke2r1
node/m2   Ready    control-plane,etcd,master   18m     v1.31.1+rke2r1
node/m3   Ready    control-plane,etcd,master   12m     v1.31.1+rke2r1
node/w1   Ready    <none>                      84s     v1.31.1+rke2r1

NAMESPACE     NAME                                                        READY   STATUS      RESTARTS        AGE
kube-system   pod/cilium-2w97z                                            1/1     Running     0               84s
kube-system   pod/cilium-b744m                                            1/1     Running     0               12m
kube-system   pod/cilium-operator-84b8ff5b6c-stfdw                        1/1     Running     4 (162m ago)    4h25m
kube-system   pod/cilium-p45b4                                            1/1     Running     1 (162m ago)    4h
kube-system   pod/cilium-qvqps                                            1/1     Running     0               18m
kube-system   pod/etcd-m1                                                 1/1     Running     1               4h32m
kube-system   pod/etcd-m2                                                 1/1     Running     0               18m
kube-system   pod/etcd-m3                                                 1/1     Running     0               11m
kube-system   pod/helm-install-rke2-cilium-z7qsz                          0/1     Completed   0               4h25m
kube-system   pod/helm-install-rke2-coredns-9cg6t                         0/1     Completed   0               4h32m
kube-system   pod/helm-install-rke2-ingress-nginx-fscw4                   0/1     Completed   0               4h32m
kube-system   pod/helm-install-rke2-metrics-server-gw8qc                  0/1     Completed   0               4h32m
kube-system   pod/helm-install-rke2-multus-j8h9d                          0/1     Completed   0               4h32m
kube-system   pod/helm-install-rke2-snapshot-controller-crd-h58vl         0/1     Completed   0               4h32m
kube-system   pod/helm-install-rke2-snapshot-controller-v6lwh             0/1     Completed   1               4h32m
kube-system   pod/helm-install-rke2-snapshot-validation-webhook-gz77q     0/1     Completed   0               4h32m
kube-system   pod/kube-apiserver-m1                                       1/1     Running     2               3h41m
kube-system   pod/kube-apiserver-m2                                       1/1     Running     0               18m
kube-system   pod/kube-apiserver-m3                                       1/1     Running     0               12m
kube-system   pod/kube-controller-manager-m1                              1/1     Running     10 (162m ago)   4h32m
kube-system   pod/kube-controller-manager-m2                              1/1     Running     0               18m
kube-system   pod/kube-controller-manager-m3                              1/1     Running     0               12m
kube-system   pod/kube-scheduler-m1                                       1/1     Running     2 (162m ago)    4h32m
kube-system   pod/kube-scheduler-m2                                       1/1     Running     0               18m
kube-system   pod/kube-scheduler-m3                                       1/1     Running     0               12m
kube-system   pod/rke2-coredns-rke2-coredns-79f8cbfcf7-9rh2l              1/1     Running     0               18m
kube-system   pod/rke2-coredns-rke2-coredns-79f8cbfcf7-bwvtc              1/1     Running     0               161m
kube-system   pod/rke2-coredns-rke2-coredns-autoscaler-7cf7db4bdc-t8ff7   1/1     Running     1 (162m ago)    4h
kube-system   pod/rke2-ingress-nginx-controller-4hns2                     1/1     Running     0               48s
kube-system   pod/rke2-ingress-nginx-controller-7c5bp                     1/1     Running     0               11m
kube-system   pod/rke2-ingress-nginx-controller-8qrjf                     1/1     Running     0               18m
kube-system   pod/rke2-ingress-nginx-controller-rwpdf                     1/1     Running     1 (162m ago)    3h40m
kube-system   pod/rke2-metrics-server-6d887864c-vdkgn                     0/1     Completed   0               4h25m
kube-system   pod/rke2-metrics-server-6d887864c-vdt5s                     1/1     Running     1 (162m ago)    4h
kube-system   pod/rke2-multus-6vcp7                                       1/1     Running     3 (50s ago)     84s
kube-system   pod/rke2-multus-bxdkm                                       1/1     Running     2 (11m ago)     12m
kube-system   pod/rke2-multus-hfqc6                                       1/1     Running     1 (162m ago)    4h
kube-system   pod/rke2-multus-t7vln                                       1/1     Running     2 (18m ago)     18m
kube-system   pod/rke2-snapshot-controller-658d97fccc-7gkrv               1/1     Running     7 (162m ago)    4h25m
kube-system   pod/rke2-snapshot-validation-webhook-784bcc6c8-xfq68        1/1     Running     1 (162m ago)    4h25m

Install K3S v1.26.9

安裝命令

$ curl -sfL https://get.k3s.io/ | INSTALL_K3S_VERSION=v1.26.9+k3s1 K3S_KUBECONFIG_MODE="644" sh -

設定 kubeconfig

$ mkdir -p $HOME/.kube && sudo cp /etc/rancher/k3s/k3s.yaml $HOME/.kube/config && sudo chown $(id -u):$(id -g) $HOME/.kube/config && sudo chmod 600 $HOME/.kube/config

Helm Install Rancher Prime v2.13.2

1. Add the Helm Chart Repository

1.1 Install helm3

在第一台 Master Node 上執行以下命令

下載 helm 壓縮檔

wget https://get.helm.sh/helm-v3.8.2-linux-amd64.tar.gz

螢幕輸出

--2022-09-21 09:06:57--  https://get.helm.sh/helm-v3.8.2-linux-amd64.tar.gz
Resolving get.helm.sh (get.helm.sh)... 152.199.39.108, 2606:2800:247:1cb7:261b:1f9c:2074:3c
Connecting to get.helm.sh (get.helm.sh)|152.199.39.108|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 13633605 (13M) [application/x-tar]
Saving to: ‘helm-v3.8.2-linux-amd64.tar.gz’

helm-v3.8.2-linux-amd64.tar.gz     100%[=============================================================>]  13.00M  5.95MB/s    in 2.2s

2022-09-21 09:07:00 (5.95 MB/s) - ‘helm-v3.8.2-linux-amd64.tar.gz’ saved [13633605/13633605]

解壓縮 helm 壓縮檔

tar zxvf helm-v3.8.2-linux-amd64.tar.gz

螢幕輸出

linux-amd64/
linux-amd64/helm
linux-amd64/LICENSE
linux-amd64/README.md

將 helm 執行檔拷貝至 PATH 環境變數中的 /usr/local/bin/ 目錄下

sudo cp linux-amd64/helm /usr/local/bin/

curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

1.2 Add the Helm Chart Repository

Prime: Recommended for production environments

helm repo add rancher-prime https://charts.rancher.com/server-charts/prime

helm repo add rancher-latest https://releases.rancher.com/server-charts/latest

helm repo add rancher-stable https://releases.rancher.com/server-charts/stable

2. Create a Namespace for Rancher

This should always be cattle-system

kubectl create ns cattle-system

3. Choose your SSL Configuration

mkdir ssl;cd ssl

下載產生自簽憑證腳本，並賦予執行權限

wget https://raw.githubusercontent.com/cooloo9871/openssl/master/mk && \
chmod +x mk

執行腳本產生自簽憑證

./mk create rancher.example.com 192.168.11.60

第一個參數放 rancher 的 Domain Name 第二個參數放 LB 的 IP (HA Rancher)，或是 rancher 本身的 IP

螢幕輸出

Generating RSA private key, 4096 bit long modulus (2 primes)
.........................................................................................................................................................................................................................++++
....................................................................................................................................................................................................................................................++++
e is 65537 (0x010001)
Generating RSA private key, 4096 bit long modulus (2 primes)
..........................................................++++
..++++
e is 65537 (0x010001)
Signature ok
subject=CN = example
Getting CA Private Key

檢查是否符合預期

ls -l
total 32
-rw------- 1 rancher users 3326 Feb 13 13:58 ca-key.pem
-rw-r--r-- 1 rancher users 2009 Feb 13 13:58 ca.pem
-rw-r--r-- 1 rancher users   41 Feb 13 13:58 ca.srl
-rw-r--r-- 1 rancher users 1582 Feb 13 13:58 cert.csr
-rw------- 1 rancher users 3243 Feb 13 13:58 cert-key.pem
-rw-r--r-- 1 rancher users 1874 Feb 13 13:58 cert.pem
-rw-r--r-- 1 rancher users   86 Feb 13 13:58 extfile.cnf
-rwxr-xr-x 1 rancher users 1166 Feb 13 13:57 mk

Create the certificate secret resource

將伺服器憑證和任何中間證書串聯到一個名為 tls.crt 的檔案中，並在一個名為 tls.key 的檔案中提供相應的證書密鑰。

kubectl -n cattle-system create secret tls tls-rancher-ingress \
--cert=cert.pem \
--key=cert-key.pem

kubectl -n cattle-system create secret generic tls-ca --from-file=cacerts.pem=./cert.pem

kubectl -n cattle-system get secret
NAME                  TYPE                                  DATA   AGE
default-token-h9lwm   kubernetes.io/service-account-token   3      2m15s
tls-ca                Opaque                                1      13s
tls-rancher-ingress   kubernetes.io/tls                     2      52s

把 ca.pem 複製到 rancher 託管的每座 Downstream cluster 中的每台 Node 上

之後使用 Rancher 註冊就不需要再勾選 insecure 了

scp ca.pem rancher@<client>:~

# ssh 到 client 上
sudo cp ca.pem /etc/pki/trust/anchors/

sudo cp ca.pem /usr/share/pki/trust/anchors/

sudo update-ca-certificates

RKE2 & Rancher Prime Installation

Preface

Install RKE2 v1.34.4

一、 安裝前環境檢查

Hardware

二、Server Node Installation

1. Config RKE2 Basic Parameters

2. Install RKE2 with v1.34.4 version

3. enable rke2 and setup kubeconfig

4. check pod status

三、Join Additional Server Nodes

1. Config RKE2 Basic Parameters

2. Install RKE2 with v1.34.4 veersion

3. enable rke2 and setup kubeconfig

4. check pod status

四、Join Agent Nodes

1. Config RKE2 Basic Parameters

2. Install RKE2 with 1.31.2 veersion

3. enable rke2

4. check pod status

Install K3S v1.26.9

Helm Install Rancher Prime v2.13.2

1. Add the Helm Chart Repository

1.1 Install helm3

1.2 Add the Helm Chart Repository

2. Create a Namespace for Rancher

3. Choose your SSL Configuration

4. Install Rancher Prime with Helm and Your Chosen Certificate Option

建立 Rancher Prime 並且使用自簽憑證

建立 Rancher Prime 並且使用 Cert-manager 產生憑證

5. Verify that the Rancher Server is Successfully Deployed

檢查名稱解析

匯出 cert-manager 簽的憑證

將憑證匯入瀏覽器

Rancher 生成 RKE2 之前的必要設定

Rancher 生成 RKE2 之後的必要設定

Helm upgrade Rancher

RKE2 設定私有 Image Registry

Rancher 設定

手動設定

Enable Rrancher Audit log

PreRequest

1. RKE2 啟動 Audit log

Step 1: Update the RKE2 Configuration File

Step 2: Create an Audit Policy File

Step 3: Restart RKE2 Server

Step 4: Monitor Audit Log

2. Enable Rrancher Audit log

Step 1: Export the current values to a file:

Step 2: 新增 Audit log 設定

Step 3: 更新 Rancher 啟動 Audit log

設定 RKE2 Cluster Pod 總量限制

Trouble shooting

在 Join Master Node 的時候，不小心設定到同一個 node name ，然後 etcd 咬住原來 node 的名稱怎麼辦

手動更新 Cert-manager 做給 Rancher 用的憑證

參考網站

一、安裝前環境檢查