Skip to content

antony@notes:~/misc$ cat "Adding-Windows-worker-nodes.md"

Adding Windows worker nodes

2025-01-20· misc

加入 Windows worker nodes

1. 開始之前

  1. 確認 Windows Server 2022 或更高版本。

打開 cmd 命令提示字元執行以下命令

systeminfo | findstr /N "." | findstr /R "^[3]:"

執行結果 :

3:作業系統名稱:         Microsoft Windows Server 2025 Standard
  1. 由 kubeadm init 建立的 kubeadm 群集。

再開一個 cmd 命令提示字元，SSH 連到 Taroko 管理主機後，執行以下命令

kubectl -n kube-system get cm kubeadm-config

執行結果 :

NAME             DATA   AGE
kubeadm-config   1      23h
  1. 確認 Containerd 版本。
kubectl -n kube-system get nodes \
  -o custom-columns="NAME:.metadata.name,VERSION:.status.nodeInfo.kubeletVersion,RUNTIME:.status.nodeInfo.containerRuntimeVersion"

執行結果 :

NAME                  VERSION   RUNTIME
tkdip-control-plane   v1.31.4   containerd://1.7.24
tkdip-worker1         v1.31.4   containerd://1.7.24
tkdip-worker2         v1.31.4   containerd://1.7.24

2. 開始加入 Windows worker nodes

針對每台機器執行下列步驟：

  1. 在機器上開啟 PowerShell。
  2. 確認使用管理員的身分執行
(New-Object Security.Principal.WindowsPrincipal([Security.Principal.WindowsIdentity]::GetCurrent())).IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)

執行結果 :

true

然後進行下列步驟。

2.1. 安裝 containerd

  1. 要安裝 containerd，首先執行以下命令：
curl.exe -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/Install-Containerd.ps1
  1. 然後執行以下命令，但首先要將 CONTAINERD_VERSION 替換為 containerd 倉庫中的最新版本。 版本必須沒有 v 前綴。 例如，使用 1.7.22 而不是 v1.7.22
.\Install-Containerd.ps1 -ContainerDVersion 1.7.24
  • 根據需要調整 Install-Containerd.ps1 的其他參數，例如 netAdapterName。
  • 如果機器不支援 Hyper-V，無法承載 Hyper-V 隔離容器，請設定 skipHypervisorSupportCheck。
  • 如果變更 Install-Containerd.ps1 可選參數 CNIBinPath 和/或 CNIConfigPath，需要使用匹配的值設定已安裝的 Windows CNI plugin。

執行結果 :

警告: Windows feature 'Containers' is not installed. Installing it now...

Success Restart Needed Exit Code      Feature Result
------- -------------- ---------      --------------
True    Yes            SuccessRest... {容器}
警告: 您必須重新啟動此伺服器才能完成安裝程序。
警告: Windows feature 'Hyper-V' is not installed. Installing it now...
True    Yes            SuccessRest... {Hyper-V}
警告: 您必須重新啟動此伺服器才能完成安裝程序。
警告: Windows feature 'Hyper-V-PowerShell' is not installed. Installing it now...
True    Yes            SuccessRest... {適用於 Windows PowerShell 的 Hyper-V 模組...
警告: 您必須重新啟動此伺服器才能完成安裝程序。
Please reboot and re-run this script.
  1. 重新開機
shutdown /r /t 1
  1. 再次執行安裝腳本
.\Install-Containerd.ps1 -ContainerDVersion 1.7.24

執行結果 :

Windows feature 'Containers' is already installed.
Windows feature 'Hyper-V' is already installed.
Windows feature 'Hyper-V-PowerShell' is already installed.
Getting ContainerD binaries
Downloading https://github.com/containerd/containerd/releases/download/v1.7.24/containerd-1.7.24-windows-amd64.tar.gz to C:\Program Files\containerd\containerd.tar.gz
x containerd.exe
x containerd-shim-runhcs-v1.exe
x containerd-stress.exe
x ctr.exe
Registering ContainerD as a service
Starting ContainerD service
Downloading https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.27.0/crictl-v1.27.0-windows-amd64.tar.gz to C:\Program Files\containerd\crictl.tar.gz
x crictl.exe


    目錄: C:\Users\bigred


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----       2025/1/20  下午 04:17                .crictl
Done - please remember to add '--cri-socket "npipe:////./pipe/containerd-containerd"' to your kubeadm join command if your kubernetes version is below 1.25!

2.2. 安裝 kubeadm and kubelet

  1. 執行下列指令以安裝 kubeadm 和 kubelet：
curl.exe -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/PrepareNode.ps1
.\PrepareNode.ps1 -KubernetesVersion v1.31.4

執行結果 :

:::spoiler

PS C:\Users\bigred> curl.exe -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/PrepareNode.ps1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  7626  100  7626    0     0  12349      0 --:--:-- --:--:-- --:--:-- 12359
PS C:\Users\bigred> .\PrepareNode.ps1 -KubernetesVersion v1.31.4
Using Kubernetes version: v1.31.4


    目錄: C:\


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----       2025/1/20  下午 04:27                k
Downloading https://dl.k8s.io/v1.31.4/bin/windows/amd64/kubelet.exe to C:\k\kubelet.exe
Downloading https://dl.k8s.io/v1.31.4/bin/windows/amd64/kubeadm.exe to C:\k\kubeadm.exe


    目錄: C:\var\log


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----       2025/1/20  下午 04:28                kubelet


    目錄: C:\var\lib\kubelet\etc


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----       2025/1/20  下午 04:28                kubernetes


    目錄: C:\etc\kubernetes


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----       2025/1/20  下午 04:28                pki
d-----       2025/1/20  下午 04:28                manifests


    目錄: C:\var\lib\kubelet\etc\kubernetes


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d----l       2025/1/20  下午 04:28                pki
Installing nssm


    目錄: C:\Program Files


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----       2025/1/20  下午 04:28                nssm
Downloading https://k8stestinfrabinaries.blob.core.windows.net/nssm-mirror/nssm-2.24.zip to nssm.zip
x nssm.exe
Registering kubelet service
Service "kubelet" installed successfully!
Set parameter "AppStdout" for service "kubelet".
Set parameter "AppStderr" for service "kubelet".
Set parameter "AppRotateFiles" for service "kubelet".
Set parameter "AppRotateOnline" for service "kubelet".
Set parameter "AppRotateSeconds" for service "kubelet".
Set parameter "AppRotateBytes" for service "kubelet".
Set parameter "DependOnService" for service "kubelet".

Caption                       :
Description                   :
ElementName                   : kubelet
InstanceID                    : kubelet
CommonName                    :
PolicyKeywords                :
Enabled                       : True
PolicyDecisionStrategy        : 2
PolicyRoles                   :
ConditionListType             : 3
CreationClassName             : MSFT|FW|FirewallRule|kubelet
ExecutionStrategy             : 2
Mandatory                     :
PolicyRuleName                :
Priority                      :
RuleUsage                     :
SequencedActions              : 3
SystemCreationClassName       :
SystemName                    :
Action                        : Allow
Direction                     : Inbound
DisplayGroup                  :
DisplayName                   : kubelet
EdgeTraversalPolicy           : Block
EnforcementStatus             : NotApplicable
LocalOnlyMapping              : False
LooseSourceMapping            : False
Owner                         :
PackageFamilyName             :
Platforms                     : {}
PolicyAppId                   :
PolicyStoreSource             : PersistentStore
PolicyStoreSourceType         : Local
PrimaryStatus                 : OK
Profiles                      : 0
RemoteDynamicKeywordAddresses : {}
RuleGroup                     :
Status                        : 已從存放區成功剖析規則。 (65536)
StatusCode                    : 65536
PSComputerName                :
Name                          : kubelet
ID                            : kubelet
Group                         :
Profile                       : Any
Platform                      : {}
LSM                           : False

Please remember that after you have joined the node to the cluster, that you have to apply the cni daemonset/service and the kube-proxy
Also remember that for kube-proxy you have to change the its version from the name of the image in the kube-proxy.yml to that of your kubernetes version

In case you use rancher, use the following commands:
For Windows you can use the following command:
curl.exe -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/kubeadm/kube-proxy/kube-proxy.yml
(Get-Content "kube-proxy.yml") -Replace 'VERSION', 'v1.31.4' | Set-Content "kube-proxy.yml"

For Linux, you can use the following command:
curl -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/kubeadm/kube-proxy/kube-proxy.yml
sed -i 's/VERSION/v1.31.4/g' kube-proxy.yml

In case you use flannel, use the following commands:
For Windows you can use the following command:
curl.exe -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/flannel/kube-proxy/kube-proxy.yml
(Get-Content "kube-proxy.yml") -Replace 'image: (.*):(.*)-(.*)-(.*)$', 'image: $1:v1.31.4-$3-$4' | Set-Content "kube-proxy.yml"

For Linux, you can use the following command:
curl -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/flannel/kube-proxy/kube-proxy.yml
sed -i -E 's/image: (.*):(.*)-(.*)-(.*)$/image: \1:v1.31.4-\3-\4/g' kube-proxy.yml

In case you use calico, use the following commands:
For Windows you can use the following command:
curl.exe -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/calico/kube-proxy/kube-proxy.yml
(Get-Content "kube-proxy.yml") -Replace 'image: (.*):(.*)-(.*)-(.*)$', 'image: $1:v1.31.4-$3-$4' | Set-Content "kube-proxy.yml"

For Linux, you can use the following command:
curl -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/calico/kube-proxy/kube-proxy.yml
sed -i -E 's/image: (.*):(.*)-(.*)-(.*)$/image: \1:v1.31.4-\3-\4/g' kube-proxy.yml

:::

  1. 產生 kubeadm join 指令
# 透過 cn 變數定義叢集名稱 (ClusterName)
cn=tkdip

# 透過 join 變數定義 worker node 加入叢集所需之命令
join=$(sudo podman exec -it "${cn}"-control-plane bash -c 'kubeadm token create --print-join-command' | tr -d '\r')

# 將字串轉換為陣列（以空格分割）
arr=($join)

# 替換第三個元素 (陣列以 0 為基礎，因此索引 2 是第三個欄位)
arr[2]="${IP}:$(sudo podman port ${cn}-control-plane | grep 6443 | cut -d ":" -f 2 )"

# 從陣列重建字串
join="${arr[@]}"

# 在螢幕印出 worker node 加入叢集所需之命令
echo $join

執行結果 :

kubeadm join 192.168.11.31:32240 --token bn7c36.hwi82i6yqvvlfs7w --discovery-token-ca-cert-hash sha256:c221fb40cf07b77620115fd5544506aad1e27055c16731d9ca52fdc7bca145db
  1. 在 Windows 執行 kubeadm join 指令
kubeadm join 192.168.11.31:32240 --token bn7c36.hwi82i6yqvvlfs7w --discovery-token-ca-cert-hash sha256:c221fb40cf07b77620115fd5544506aad1e27055c16731d9ca52fdc7bca145db

執行結果 :

local-path-provisioner sharedFileSystemPath examplePod Priority and Preemption