Antony 筆記：技術原理與實務經驗 – 虛擬化

部署 VKS Clusters

Tue, 10 Feb 2026 05:10:28 +0800

1. 先決條件

多個 ESX 主機透過 vSphere Distributed Switch (VDS) 連接。
已設定 Shared storage，例如 vSAN 或 NFS。
已啟用 vSphere HA 和 fully-automated DRS。
已啟用 vSphere Lifecycle Manager (vLCM)。
已設定網路，可使用 NSX 及其內嵌的 load balancer，或使用 VDS 搭配外部的 load balancer。

2. ESXI 安裝設定

幫每一台 esxi 都設定固定 ip、DNS

幫每一台 esxi 設定 ntp 校時

2.1 設定 vCenter

參考：https://hackmd.io/@wu-andy/rkXpmTGteg

2.2. 環境資訊

vCenter 版本：8.0.3
3 台 ESXi 版本：8.0.3

2.3. 設定 VSS 和 VDS

每台 ESXI 要再添加兩張網卡，總共三個，並且都要不同網段

重新命名 VSS(VM Network)

此網段為 172.21.1.0/24
將 VSS(VM Network) 重新命名成 mgmt，所有 ESXI 都要做，這裡更改名稱主要是因為 supervisor 再找網卡會因為名稱有空格找不到

第二台 ESXI

第三台 ESXI

設定 VDS 給 VIP 使用

此網段為 172.20.2.0/24
做出 VDS ，這個 VDS 給 VIP 使用

在 vSphere Client 中，導覽到 Datacenter。
在瀏覽器中的 Datacenter 上按一下滑鼠右鍵，然後選取 Distributed Switch > 新增 Distributed Switch。
輸入新 Distributed Switch 的名稱，例如：DSwitch。
在選取版本中，輸入 Distributed Switch 的版本。選取 8.0.3。
在設定組態中，輸入上行連接埠數目，輸入值 1 和取名 VIP。
檢閱設定，然後按一下完成。
在您建立的 Distributed Switch 上按一下滑鼠右鍵，然後選取設定編輯設定。
在進階索引標籤上，輸入大於 1700 的值作為 MTU (位元組) 值，然後按一下確定。在承載覆疊流量的任何網路上，MTU 大小必須大於或等於 1700。例如，9000。如果 vks 是安裝在 NSX 環境， MTU 就要設大於 1700。沒有的話可以只設 1500
加入 esxi 主機

指定一張網卡

VDS 給 workload cluster 使用

此網段為 172.20.3.0/24
再做出一個 VDS，這個 VDS 給 workload cluster 使用

加入 esxi 主機

指定一張網卡

2.4. 確認已設定 vsan

要記得略過快速入門

在監控 -> Skyline Health 點選無回應警示

2.5. 確認已啟用 vSphere HA 和 fully-automated DRS

2.6. 確認已啟用 vSphere Lifecycle Manager (vLCM)

安裝 Avi VM

VMware Avi Load Balancer

設定 Avi

第一次登入才需設管理帳號密碼

設定一組密碼，用來加密設定匯出與排程備份中的敏感資料（例如金鑰、憑證、雲端/整合帳密等）。之後做還原/匯入時，必須提供同一組密語，否則無法還原。

設定 avi 自己的 vip

如果有多個 avi 主機也可以把其他主機加入，做出 avi 叢集

可以透過 vip 登入 avi

設定授權

到授權頁面把 license 改成 enterprise tier

設定 DNS/NTP

Administration Settings DNS/NTP

設定 VMware Cloud

建立 vCenter Cloud 讓 Avi 能「在 vSphere 上自動部署並管理」Service Engine，並使用該 vSphere 環境的 network/storage/compute 資源來提供 load balancing

透過在「建立」下拉清單中選擇此選項來設定VMware vCenter/vSphere ESX雲端

為雲端提供一個名稱，然後從類型選單中選擇 VMware vCenter/vSphere ESX

設定關閉 dhcp 和 ipv6
勾選 Prefer Static Routes vs Directly Connected Network : Avi 在決定如何把 VIP 的流量送出去時，會優先使用「靜態路由（static route）」定義的 next-hop，而不是直接利用 SE 所在介面上的「直接相連（directly-connected）」行為（例如 ARP）去處理該子網流量

配置 vCenter 的憑證

選擇指定的 datacenter ，以及選擇 Service Engine（SE）用來做管理介面（management vNIC）的網路。不用勾選 content library，然後保存

設定 Network

設定後保存離開，等待狀態變為綠色

Infrastructure > Cloud Resources > Networks

定義一個子網路（subnet）與一組靜態 IP 範圍，供 Controller 配發給 Virtual Service 的 VIP（虛擬 IP）和 Service Engine（SE）VM 的管理 / data IP

這邊定義 ip range，supervisor 在啟用的時候會先從裡面拿兩個 VIP 出來用，這兩個 VIP 後端會導流到 supervisor 在 worklaod 網路的那隻腳

所以 supervisor 會有三個 VIP

一個在管理網段(5 個 IP 裡面其中的一個) —> 提供 supervisor 內部運作用

兩個在 SE 的 VI P範圍 (獨立的兩個VIP) —> 提供使用這下指令與監控用 1 號不過 AVI ，由 supervisor CP 節點取得 2 號使用 AVI，在啟用過程中由 vCenter/Supervisor 發送 API 自動配置

定義 VIP ip range

建立 IPAM profile

建立虛擬服務時，需要使用 IPAM 來指派虛擬 IP 位址。

當有人（例如 vCenter Supervisor）向我要一個 VIP 時，我被授權從哪些網路發放 IP？這邊指定 VIP 的 network。

建立 DNS profile

這邊先新增任意一個 domain

再回到 VMware Cloud 設定 IPAM

設定 Certificate

這邊 common name 要填的是 avi 自己的 vip 所對應的 domain

SAN 要填的就是 avi 自己的 vip 位置

複製憑證，待會在創建 supervisor cluster 時會需要貼上此憑證

更換訪問 avi 入口憑證

設定 Service Engine Group

Infrastructure > Cloud Resources > Service Engine Group

設定 Static Routes

Infrastructure > Cloud Resources > VRF Context

這邊要設定 workload 網段要路由到 VIP 網段的 gateway

設定 vSphere zone

選擇要用於 VKS 叢集和 vSphere Pod 的叢集

設定工作負載管理 (啟動 supervisor)

要將前面在 avi 創建的憑證放進來

起始 ip 這邊是設給 supervisor cluster 的 vip，他需要保留五個連續 ip ，所以這裡設定 172.21.1.30，就是代表到 172.21.1.34 都會分配給 supervisor cluster 使用，會使用的是 supervisor control plane 的節點 IP+VIP+升級保留，總共 5 個 IP，但是使用順序不一定
這意這邊起始 ip 不能跟前面 avi 設定的 vip 範圍重疊

設定 workload cluster 網路資訊

這邊會設定 workload cluster 的 vm 他拿到的 ip 是在以下範圍
網路名稱要填小寫
網路範圍也不能跟前面重複

創建 workload cluster(TKG)

確認 supervisor 狀態都正確

開啟瀏覽器，連接 https://172.21.2.81，下載 kubectl CLI Plugin

準備一台 ubuntu bastion 機使用指令下載

$ curl -k https://172.21.2.81/wcp/plugin/linux-amd64/vsphere-plugin.zip -o vsphere-plugin.zip

解壓縮

$ sudo apt install unzip
$ unzip vsphere-plugin.zip
$ sudo cp -r bin/* /usr/local/bin

登入 supervisor，使用 VC 帳號登入

$ kubectl vsphere login --insecure-skip-tls-verify --server=172.21.2.81
Username: administrator@vsphere.local
Password:
Logged in successfully.
You have access to the following contexts:
172.21.2.81
svc-tkg-domain-c19
svc-velero-domain-c19
If the context you wish to use is not in this list, you may need to try
logging in again later, or contact your cluster administrator.
To change context, use `kubectl config use-context <workload name>`

此時看到的 control-plane 三台就是剛剛創建的 supervisor vm，而 worker 就是 esxi 三台，因此如果在這裡創見 pod ，就是直接跑在 esxi 上

$ kubectl get no -owide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
423d9b295547c821ffd05433fe560885 Ready control-plane,master 42m v1.30.10+vmware.1-fips 172.21.3.52 <none> VMware Photon OS/Linux 5.10.237-2.ph4-esx containerd://1.7.25+vmware.2-fips
423dee48377f7b6bf5ee23a17f104118 Ready control-plane,master 42m v1.30.10+vmware.1-fips 172.21.3.51 <none> VMware Photon OS/Linux 5.10.237-2.ph4-esx containerd://1.7.25+vmware.2-fips
423df53a268cb03c9c8f7833d1968033 Ready control-plane,master 50m v1.30.10+vmware.1-fips 172.21.3.50 <none> VMware Photon OS/Linux 5.10.237-2.ph4-esx containerd://1.7.25+vmware.2-fips
esxi1.andy.lab Ready agent 4m58s v1.30.5-sph-806add6 172.21.1.2 <none> <unknown> <unknown> <unknown>
esxi2.andy.lab Ready agent 7m v1.30.5-sph-806add6 172.21.1.3 <none> <unknown> <unknown> <unknown>
esxi3.andy.lab Ready agent 5m53s v1.30.5-sph-806add6 172.21.1.4 <none> <unknown> <unknown> <unknown>

建立 vSphere namespace

建立一個 tkg-workloads namespace

新增權限

設定 administrator@vsphere.local 為擁有者

新增儲存區

vSAN Default Storage Policy 在 sc 裡名稱會叫 vsan-default-storage-policy

$ kubectl get storageclass
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
management-storage-policy-encryption csi.vsphere.vmware.com Delete Immediate true 5d18h
management-storage-policy-regular csi.vsphere.vmware.com Delete Immediate true 5d18h
management-storage-policy-single-node csi.vsphere.vmware.com Delete Immediate true 5d18h
management-storage-policy-stretched-lite csi.vsphere.vmware.com Delete Immediate true 5d18h
management-storage-policy-thin csi.vsphere.vmware.com Delete Immediate true 5d18h
vm-encryption-policy csi.vsphere.vmware.com Delete Immediate true 5d18h
vsan-default-storage-policy csi.vsphere.vmware.com Delete Immediate true 5d18h

新增 VM Class

選用預設定義的 best-effort-medium

也可以建立自定義 VM Class

建立一個 4core CPU,16G RAM

登入 supervisor，使用 VC 帳號登入

$ kubectl vsphere login --insecure-skip-tls-verify --server=172.21.2.81
Username: administrator@vsphere.local
Password:
Logged in successfully.
You have access to the following contexts:
10.200.2.31
svc-tkg-domain-c17
svc-velero-domain-c17
tkg-workloads

切換 context

$ kubectl config use-context tkg-workloads

檢視目前 TKG 可使用的版本，這邊會使用 v1.32.3---vmware.1-fips-vkr.2 版本

$ kubectl get tkr
v1.31.11---vmware.3-fips-vkr.1 v1.31.11+vmware.3-fips-vkr.1 True True 5d17h
v1.31.4---vmware.1-fips-vkr.3 v1.31.4+vmware.1-fips-vkr.3 True True 5d17h
v1.31.7---vmware.1-fips-vkr.2 v1.31.7+vmware.1-fips-vkr.2 True True 5d17h
v1.32.0---vmware.6-fips-vkr.2 v1.32.0+vmware.6-fips-vkr.2 True True 5d17h
v1.32.3---vmware.1-fips-vkr.2 v1.32.3+vmware.1-fips-vkr.2 True True 5d17h
v1.32.7---vmware.3-fips-vkr.1 v1.32.7+vmware.3-fips-vkr.1 True True 5d17h
......

在 content library 檢查是否已經存在

如果遇到一直同步失敗，就移除 Kubernetes Service Content Library 後再重新指定讓他在同步一次

離線環境要自行準備 content library https://wp-content.vmware.com/v2/latest/ 可參考

編輯 cluster.yaml

$ nano my-tkg-cluster.yaml
apiVersion: run.tanzu.vmware.com/v1alpha2
kind: TanzuKubernetesCluster
metadata:
name: tkgs-v2-cluster-default
namespace: tkg-workloads
spec:
topology:
controlPlane:
replicas: 1
vmClass: best-effort-medium # 根據自己定義的名稱更換
storageClass: 'vsan-default-storage-policy' # 根據自己定義的名稱更換
tkr:
reference:
name: v1.32.3---vmware.1-fips-vkr.2
nodePools:
- name: worker-nodepool-a1
replicas: 1
vmClass: best-effort-medium # 根據自己定義的名稱更換
storageClass: 'vsan-default-storage-policy' # 根據自己定義的名稱更換
tkr:
reference:
name: v1.32.3---vmware.1-fips-vkr.2

部署 3m1w 架構

apiVersion: run.tanzu.vmware.com/v1alpha2
kind: TanzuKubernetesCluster
metadata:
name: my-tkgs-cluster
namespace: tkg-workloads
spec:
topology:
controlPlane:
replicas: 3
vmClass: myvm # 根據自己定義的名稱更換
storageClass: 'vsan-default-storage-policy' # 根據自己定義的名稱更換
tkr:
reference:
name: v1.32.3---vmware.1-fips-vkr.2
nodePools:
- name: worker-nodepool-a1
replicas: 1
vmClass: myvm # 根據自己定義的名稱更換
storageClass: 'vsan-default-storage-policy' # 根據自己定義的名稱更換
volumes:
- name: containerd
mountPath: /var/lib/containerd
capacity:
storage: 50Gi
tkr:
reference:
name: v1.32.3---vmware.1-fips-vkr.2
settings:
storage:
defaultClass: 'vsan-default-storage-policy'
network:
cni:
name: calico
serviceDomain: cluster.local

新版使用 cluster yaml 部署 TKG 參考

部署 TKG

$ kubectl apply -f my-tkg-cluster.yaml

檢視 TKG 部署完成

$ kubectl -n tkg-workloads get tkc
Warning: run.tanzu.vmware.com/tanzukubernetescluster is deprecated. It is succeeded by cluster.x-k8s.io/cluster. Refer to documentation.
NAME CONTROL PLANE WORKER KUBERNETES RELEASE NAME AGE READY KUBERNETES RELEASE COMPATIBLE UPDATES AVAILABLE
tkgs-v2-cluster-default 1 1 v1.32.7---vmware.3-fips-vkr.1 80m True True

登入 TKG 叢集

$ kubectl vsphere login --server=10.200.2.31 \
--tanzu-kubernetes-cluster-name tkgs-v2-cluster-default \
--tanzu-kubernetes-cluster-namespace tkg-workloads \
--insecure-skip-tls-verify \
-u administrator@vsphere.local

檢視叢集狀態是否正常

$ kubectl get no
NAME STATUS ROLES AGE VERSION
tkgs-v2-cluster-default-ljb66-8jk74 Ready control-plane 18m v1.32.7+vmware.3-fips
tkgs-v2-cluster-default-worker-nodepool-a1-jh2cz-f65qq-9q5fl Ready <none> 9m1s v1.32.7+vmware.3-fips
$ kubectl get po -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system antrea-agent-9dqxv 2/2 Running 0 9m47s
kube-system antrea-agent-tjkn5 2/2 Running 0 17m
kube-system antrea-controller-798b767987-7hl8h 1/1 Running 0 17m
kube-system coredns-5958c95bcd-hvdrg 1/1 Running 0 18m
kube-system coredns-5958c95bcd-ms2qz 1/1 Running 0 15m
kube-system docker-registry-tkgs-v2-cluster-default-ljb66-8jk74 1/1 Running 0 19m
kube-system docker-registry-tkgs-v2-cluster-default-worker-nodepool-a1-jh2cz-f65qq-9q5fl 1/1 Running 0 9m47s
kube-system etcd-tkgs-v2-cluster-default-ljb66-8jk74 1/1 Running 0 19m
kube-system kube-apiserver-tkgs-v2-cluster-default-ljb66-8jk74 1/1 Running 0 19m
kube-system kube-controller-manager-tkgs-v2-cluster-default-ljb66-8jk74 1/1 Running 1 (17m ago) 19m
kube-system kube-proxy-lt8qd 1/1 Running 0 9m47s
kube-system kube-proxy-rfmtp 1/1 Running 0 18m
kube-system kube-scheduler-tkgs-v2-cluster-default-ljb66-8jk74 1/1 Running 1 (17m ago) 19m
kube-system metrics-server-54d7fc854c-5v4zn 1/1 Running 0 17m
kube-system snapshot-controller-fd845b8f8-fwxr2 1/1 Running 0 18m
secretgen-controller secretgen-controller-5dd46bbcfc-tljsr 1/1 Running 0 17m
tkg-system kapp-controller-7449dcf686-d2hnt 2/2 Running 0 18m
vmware-system-antrea antrea-pre-upgrade-job-klmv4 0/1 Completed 0 18m
vmware-system-auth guest-cluster-auth-svc-vz9zh 1/1 Running 0 16m
vmware-system-cloud-provider guest-cluster-cloud-provider-86466bf878-jp8t2 1/1 Running 1 (17m ago) 18m
vmware-system-csi vsphere-csi-controller-798c4f56b9-gcr69 7/7 Running 0 18m
vmware-system-csi vsphere-csi-node-fqs8t 3/3 Running 4 (15m ago) 18m
vmware-system-csi vsphere-csi-node-r6csv 3/3 Running 0 9m47s

troubshooting

進到 VC

root@vc [ ~ ]# cd /usr/lib/vmware-wcp/
# 查看 supervisor 密碼
root@vc [ /usr/lib/vmware-wcp ]# ./decryptK8Pwd.py
Read key from file
Connected to PSQL
Cluster: domain-c17:16f96da8-eab0-4cb7-b70c-fdcb7737bb3a
IP: 10.200.0.60
PWD: (WV92wWtP{+7F^~1
------------------------------------------------------------

登入 supervisor 後查看 pod 是否異常

root@422f993f634f1e829e2340fa1d73b0eb [ ~ ]# k -n vmware-system-ako get po
NAME READY STATUS RESTARTS AGE
vmware-system-ako-ako-controller-manager-677ff55d9d-x5k94 1/1 Running 0 53m

參考

https://techdocs.broadcom.com/us/en/vmware-security-load-balancing/avi-load-balancer/avi-load-balancer-vmware-cloud-foundation/9-0/deploying-vsphere-supervisor-with-vds-and-avi-load-balancer.html

https://techdocs.broadcom.com/us/en/vmware-cis/vcf/vsphere-supervisor-services-and-standalone-components/latest/managing-vsphere-kubernetes-service/running-tkg-service-clusters/deploying-tkg-service-clusters.html

https://techdocs.broadcom.com/us/en/vmware-security-load-balancing/avi-load-balancer/avi-load-balancer/22-1/vmware-avi-load-balancer-administration-guide/access-settings/certificates/changing-the-nsx-advanced-load-balancer-controller-s-default-certificate.html

Set up VMware Aria Operations configuration for third-party Prometheus exporters

Fri, 14 Nov 2025 08:12:42 +0800

The steps to set up the configuration file for the first time are as follows:

點擊左側選單 -> Operations -> Configuration -> Management packs Configuration
點擊 ADD

填入以下內容：

Name: k8s-custom-prom-metric-mappings.xml 注意，設定檔的名字不能更改，只能設定這名字

設定值如下：

<MappingDefinitions
 xmlns:xs="http://www.w3.org/2001/XMLSchema">
 <CustomMetricConfig collectCustomMetrics="true"
 enableAllLabels="false"
 labelNamesToExclude="beta_kubernetes_io_os,container_label_annotation_io_kubernetes_container_hash,container_label_annotation_io_kubernetes_container_ports,container_label_annotation_io_kubernetes_container_restartCount,container_label_annotation_io_kubernetes_container_terminationMessagePath,container_label_annotation_io_kubernetes_container_terminationMessagePolicy,container_label_annotation_io_kubernetes_pod_terminationGracePeriod,container_label_annotation_kubeadm_kubernetes_io_etcd_advertise_client_urls,container_label_annotation_kubeadm_kubernetes_io_kube_apiserver_advertise_address_endpoint,container_label_annotation_kubernetes_io_config_hash,container_label_annotation_kubernetes_io_config_seen,container_label_annotation_kubernetes_io_config_source,container_label_app,container_label_app_kubernetes_io_name,container_label_app_kubernetes_io_version,container_label_component,container_label_controller_revision_hash,container_label_description,container_label_io_kubernetes_container_logpath,container_label_io_kubernetes_docker_type,container_label_io_kubernetes_pod_uid,container_label_io_kubernetes_sandbox_id,container_label_k8s_app,container_label_maintainer,container_label_maintainers,container_label_pod_template_generation,container_label_pod_template_hash,container_label_purpose,container_label_tier,container_label_version,container_runtime_version,dockerVersion,goversion,image,image_id,instance,timezone,container_label_io_kubernetes_container_name,container_label_io_kubernetes_pod_name,container_label_io_kubernetes_pod_namespace,name"
 maxGroupNames="3"></CustomMetricConfig>
 <IdentifierMappings>
 <NodeIdentifierMapping
 labelContainingNodeName="nodename,node_name,node,kubernetes_node_name" />
 <NamespaceIdentifierMapping
 labelContainingNamespaceName="namespace,kubernetes_namespace,container_label_io_kubernetes_pod_namespace,source_workload_namespace" />
 <PodIdentifierMapping
 labelContainingPodName="pod,pod_name,podname,container_label_io_kubernetes_pod_name" />
 <ContainerIdentifierMapping
 labelContaingContainerID="id,container_id"
 labelContainingContainerName="container_name,container" />
 <DeploymentIdentifierMapping
 labelContainingDeploymentName="deployment" />
 <ReplicasetIdentifierMapping
 labelContainingReplicasetName="replicaset" />
 </IdentifierMappings>
 <MappingDefinition resourceKind="namespace">
 <MetricGroup groupName="CXS-PVC-Prometheus" isInstanced="true">
 <Metric instancedLabelName="persistentvolumeclaim"
 metricName="Persistent Volume Claim Size in Bytes"
 metricUnit="Bytes"
 isRate="false"
 rule='kube_persistentvolumeclaim_resource_requests_storage_bytes{namespace="$NAMESPACE_NAME"}'
 ruleName='persistentvolumeclaim'/>
 </MetricGroup>
 <MetricGroup groupName="CXS-ETCD-Prometheus" isInstanced="true">
 <Metric instancedLabelName="operation,resource"
 metricName="Etcd Request Duration Seconds Sum"
 metricUnit=""
 isRate="false"
 rule='etcd_request_duration_seconds_sum'
 ruleName='etcd_request_duration_seconds_sum'/>
 <Metric instancedLabelName="operation,resource"
 metricName="Etcd Request Duration Seconds Count"
 metricUnit=""
 isRate="false"
 rule='etcd_request_duration_seconds_count'
 ruleName='etcd_request_duration_seconds_count'/>
 <Metric instancedLabelName="operation,resource"
 metricName="ETCD Average Per Request Latency"
 metricUnit="sec"
 isRate="true"
 rule='rate(etcd_request_duration_seconds_sum[5m]) / rate(etcd_request_duration_seconds_count[5m])'
 ruleName='etcd_average_request_latency'/>
 <Metric instancedLabelName=""
 metricName="ETCD Average Request Latency"
 metricUnit="sec"
 isRate="true"
 rule='sum(rate(etcd_request_duration_seconds_sum[5m])) / sum(rate(etcd_request_duration_seconds_count[5m]))'
 ruleName='etcd_average_request_latency'/>
 <Metric instancedLabelName="operation,resource"
 metricName="Etcd Request Duration Seconds bucket"
 metricUnit=""
 isRate="false"
 rule='etcd_request_duration_seconds_bucket'
 ruleName='etcd_request_duration_seconds_bucket'/>
 <Metric instancedLabelName=""
 metricName="ETCD P95 Request Duration"
 metricUnit=""
 isRate="false"
 rule='histogram_quantile(0.95, sum by (le) (rate(etcd_request_duration_seconds_bucket[5m])))'
 ruleName='etcd_p95_request_duration'/>
 <Metric instancedLabelName=""
 metricName="ETCD P99 Request Duration"
 metricUnit=""
 isRate="false"
 rule='histogram_quantile(0.99, sum by (le) (rate(etcd_request_duration_seconds_bucket[5m])))'
 ruleName='etcd_p99_request_duration'/>
 <Metric instancedLabelName=""
 metricName="ETCD Request Error Rate (%)"
 metricUnit=""
 isRate="true"
 rule='( sum(rate(etcd_request_errors_total[5m])) / sum(rate(etcd_requests_total[5m])) ) * 100'
 ruleName='etcd_request_error_rate'/>
 <Metric instancedLabelName=""
 metricName="ETCD Request Rate (RPS)"
 metricUnit=""
 isRate="false"
 rule='sum(rate(etcd_requests_total[1m]))'
 ruleName='etcd_request_rate'/>
 </MetricGroup>
 <MetricGroup groupName="CXS-API-Server-Prometheus" isInstanced="true">
 <Metric instancedLabelName=""
 metricName="API Server Request Execution Duration (p95)"
 metricUnit=""
 isRate="false"
 rule='histogram_quantile(0.95, sum by (le) (rate(apiserver_flowcontrol_request_execution_seconds_bucket[5m])))'
 ruleName='api_server_request_execution_duration_p95'/>
 <Metric instancedLabelName=""
 metricName="API Server Request Execution Duration (p99)"
 metricUnit=""
 isRate="false"
 rule='histogram_quantile(0.99, sum by (le) (rate(apiserver_flowcontrol_request_execution_seconds_bucket[5m])))'
 ruleName='api_server_request_execution_duration_p99'/>
 <Metric instancedLabelName=""
 metricName="API Server Request Wait Duration (p95)"
 metricUnit=""
 isRate="false"
 rule='histogram_quantile(0.95, sum by (le) (rate(apiserver_flowcontrol_request_wait_duration_seconds_bucket[5m])))'
 ruleName='api_server_request_wait_duration_p95'/>
 <Metric instancedLabelName=""
 metricName="API Server Request Wait Duration (p99)"
 metricUnit=""
 isRate="false"
 rule='histogram_quantile(0.99, sum by (le) (rate(apiserver_flowcontrol_request_wait_duration_seconds_bucket[5m])))'
 ruleName='api_server_request_wait_duration_p99'/>
 <Metric instancedLabelName=""
 metricName="API Server 5xx Error Rate (%)"
 metricUnit=""
 isRate="false"
 rule='(sum(rate(apiserver_flowcontrol_dispatched_requests_total{code=~"5.."}[5m])) or vector(0)) / sum(rate(apiserver_flowcontrol_dispatched_requests_total[5m])) * 100'
 ruleName='api_server_5xx_error_rate'/>
 <Metric instancedLabelName=""
 metricName="API Server In-flight Requests"
 metricUnit=""
 isRate="false"
 rule='100 * sum(apiserver_current_inflight_requests) / clamp_min(max_over_time((sum(apiserver_current_inflight_requests))[7d:5m]), 1)'
 ruleName='api_server_in_flight_requests'/>
 <Metric instancedLabelName=""
 metricName="API Server Long-running Requests"
 metricUnit=""
 isRate="false"
 rule='100 * sum(apiserver_longrunning_requests) / avg_over_time((sum(apiserver_longrunning_requests))[7d:5m])'
 ruleName='api_server_long_running_requests'/>
 </MetricGroup>
 <MetricGroup groupName="CXS-CoreDNS-Prometheus" isInstanced="true">
 <Metric instancedLabelName=""
 metricName="CoreDNS DNS Query P95 Latency"
 metricUnit=""
 isRate="false"
 rule='histogram_quantile(0.95, sum(rate(coredns_dns_request_duration_seconds_bucket[5m])) by (le))'
 ruleName='coredns-dns-query-p95-latency'/>
 <Metric instancedLabelName=""
 metricName="CoreDNS DNS Query Rate"
 metricUnit=""
 isRate="false"
 rule='sum(rate(coredns_dns_requests_total[1m]))'
 ruleName='coredns-dns-query-rate'/>
 </MetricGroup>
 </MappingDefinition>
 <MappingDefinition resourceKind="container">
 <MetricGroup groupName="CXS-Prometheus" isInstanced="true">
 <Metric instancedLabelName=""
 metricName="Status Running"
 metricUnit=""
 isRate="false"
 rule='kube_pod_container_status_running'
 ruleName='kube_pod_container_status_running'/>
 <Metric instancedLabelName=""
 metricName="Pod Restart Count"
 metricUnit=""
 isRate="false"
 rule='increase(kube_pod_container_status_restarts_total[1h])'
 ruleName='pod-restart-count'/>
 <Metric instancedLabelName=""
 metricName="Container Cpu Usage Seconds Total[5m]"
 metricUnit="sec"
 isRate="true"
 rule='sum(rate(container_cpu_usage_seconds_total{namespace="$NAMESPACE_NAME",container="$CONTAINER_NAME"}[5m])) * 100'
 ruleName='container_cpu_usage_seconds_total[5m]'/>
 </MetricGroup>
 </MappingDefinition>
 <MappingDefinition resourceKind="node">
 <MetricGroup groupName="CXS-filesystem" isInstanced="true">
 <Metric instancedLabelName="mountpoint"
 metricName="Filesystem Avail Space"
 metricUnit="Bytes" promMetricName="node_filesystem_avail_bytes" isRate="false"/>
 <Metric instancedLabelName="mountpoint"
 metricName="Filesystem Size"
 metricUnit="Bytes" promMetricName="node_filesystem_size_bytes" isRate="false"/>
 </MetricGroup>
 <MetricGroup groupName="CXS-ETCD_metrics" isInstanced="true">
 <Metric instancedLabelName=""
 metricName="ETCD Database Used Size"
 metricUnit="Bytes"
 isRate="false"
 rule='etcd_mvcc_db_total_size_in_bytes'
 ruleName='etcd_mvcc_db_total_size_in_bytes'/>
 <Metric instancedLabelName=""
 metricName="ETCD Database Backend Size"
 metricUnit="Bytes"
 isRate="false"
 rule='etcd_server_quota_backend_bytes'
 ruleName='etcd_server_quota_backend_bytes'/>
 <Metric instancedLabelName=""
 metricName="ETCD Database Usage(%)"
 metricUnit=""
 isRate="false"
 rule='(etcd_mvcc_db_total_size_in_bytes / etcd_server_quota_backend_bytes) * 100'
 ruleName='etcd_server_db_usage'/>
 </MetricGroup>
 </MappingDefinition>
</MappingDefinitions>

點擊左下角 SAVE 按鈕

讓 operations 重新套用設定檔 (1)
讓 operations 重新套用設定檔 (2)
看 log
確認有收到 Metrics

參考文件

Set up VMware Aria Operations configuration for third-party Prometheus exporters - Broadcom Docs

認識 VKS Components

Thu, 18 Sep 2025 09:55:56 +0800

VKS（先前稱為 TKG Service 或 TKGS）為 Kubernetes workload 叢集提供自助式的生命週期管理。VKS with Supervisor 已針對 vSphere 環境進行最佳化，並與底層的基礎架構整合，包括 vCenter、ESX、virtual networking 和cloud native storage。透過 VKS，您可以佈建符合規範的 Kubernetes 叢集，並維持與上游的一致性。

VKS 包含數個與 vSphere Supervisor 整合的元件。

1. vSphere Supervisor

Supervisor 是內建於 vSphere 的一個關鍵 control plane，它將「所需狀態 (desired state)」和單一的一致性 API 直接整合到 hypervisor 中，從而能夠在傳統 workloads 旁邊，以原生的方式部署與管理：

Kubernetes clusters
declarative Virtual Machines (宣告式虛擬機器)
vSphere Pod。

它將 vSphere 轉變為一個可擴展的平台，提供一個宣告式的 API 介面，讓使用者能在 vSphere Namespace 所啟用的資源邊界內，自助式地佈建現代基礎架構服務。

部署 Supervisor 的兩種方式

vSphere Cluster 部署 Supervisor 的典型方式，是將其部署在單一的 vSphere 叢集上。vSphere 叢集是由多個 ESX 主機組成的集合，並由一個 vCenter 系統進行管理。vSphere 叢集必須設定特定的功能以支援 Supervisor，包括：
- 多個 ESX 主機透過 vSphere Distributed Switch (VDS) 連接。
- 已設定 Shared storage，例如 vSAN 或 NFS。
- 已啟用 vSphere HA 和 fully-automated DRS。
- 已啟用 vSphere Lifecycle Manager (vLCM)。
- 已設定網路，可使用 NSX 及其內嵌的 load balancer，或使用 VDS 搭配外部的 load balancer。
從架構上來說，對於在 Supervisor 上進行的 VKS 生產環境部署，您應該將執行 vCenter 的管理平面主機或叢集，與將要啟用 Supervisor 的運算平面叢集分開。如果您使用一個 vSphere 叢集來託管 vCenter，該叢集不應啟用 DRS。
vSphere Zone 可以將 vSphere 叢集指派給 vSphere 區域 (Zone)，以便為 Supervisor 提供高可用性與容錯能力。將 Supervisor 跨 vSphere 區域部署，可讓您在特定的可用性區域中佈建 VKS 叢集。

在單一 vSphere 叢集上部署 Supervisor 時，Supervisor 與 vSphere 叢集之間存在一對一的關係。而在區域化的 Supervisor 部署中，Supervisor 會橫跨三個 vSphere 叢集，為其上的 VKS 叢集提供高可用性與故障網域 (failure domains)。

vSphere 管理員會建立這三個 vSphere 區域，並將 vSphere 叢集與之關聯。除了 vSphere 叢集本身的要求外，在三個 vSphere 區域上部署 Supervisor 還有一些特殊要求，包括：
- 三個 vSphere 區域透過 vSphere 分散式交換器 (VDS) 連接。
- 每個 vSphere 區域包含一個 vSphere 叢集。
- Supervisor 必須部署在正好 3 個 vSphere 區域上。
- vSphere 儲存策略 (storage policy) 可用於 vSphere 區域。

2. vSphere Namespace

vSphere namespace 是一個租用邊界 (tenancy boundary)，它能讓使用者自助式地取用 3 種 runtime，包括虛擬機器、VKS 叢集、vSphere Pods 以及其他多種基礎架構服務。

對於每個 vSphere namespace，可以設定 RBAC、persistent storage、resource limits、images library 以及虛擬機器類別。

3. VKS

VKS 是開源 Cluster API 專案的一種實作，其定義了一組自訂資源和控制器，用以管理 Kubernetes 叢集的生命週期。VKS 是 Supervisor 的一個元件。

VKS 具有三層控制器來管理 VKS 叢集的生命週期，包括 Virtual Machine Service、Cluster API 以及 Cloud Provider Plugin：

VM Operator Virtual Machine Service controller 提供了一種宣告式、Kubernetes 風格的 API，用於管理虛擬機及相關的 vSphere 資源。虛擬機服務引入了「虛擬機類別 (virtual machine class)」的概念，它代表一種抽象且可重複使用的硬體配置。VKS 使用 Virtual Machine Service 來管理託管workload cluster 的 control plane 和 worker node 虛擬機的生命週期。
Cluster API Cluster API controller 提供了宣告式、Kubernetes 風格的 API，用於叢集的建立、設定與管理。Cluster API 的輸入內容包括：
- 描述叢集的資源
- 一組描述構成叢集的虛擬機的資源
- 一組描述叢集附加元件的資源
Cloud Provider Plugin VKS 會置備 workload clusters，其中包含與底層 vSphere Namespace 資源整合所需的元件。這些元件包含一個與 Supervisor 整合的 Cloud Provider Plugin。VKS 使用此 Cloud Provider Plugin，將 persistent volumes 的請求傳遞給 Supervisor，而 Supervisor 已與 VMware Cloud Native Storage (CNS) 整合。

4. vSphere Kubernetes release

vSphere Kubernetes release (VKr) 提供了由 VMware 簽署並支援的 Kubernetes 軟體發行版和附加元件，供 VKS 叢集使用。

每個 VKr 皆以虛擬機範本 (OVA 檔案) 的形式發佈。VKS 使用 OVA 格式來建構 VKS 叢集的虛擬機節點。VKr 根據 Kubernetes 的版本控制方式進行版本管理，並包含針對 vSphere 基礎架構的作業系統客製化與最佳化。

Content Library (內容庫) vSphere Kubernetes release 是透過 vCenter content library 提供給 VKS 叢集使用。您可以建立一個訂閱的內容庫，以便在 VMware 發佈新的 VKr 版本時自動接收。

5. VKS 叢集元件

在 VKS 叢集中執行的元件為產品的四個領域提供功能：驗證、儲存、網路和負載平衡。

5.1. Authentication Webhook

Authentication Webhook 以 Pod 的形式在叢集內部執行，用以驗證使用者驗證 token。
VKS 叢集支援兩種驗證方式：使用 vCenter Single Sign-On，以及使用支援 Open ID Connect (OIDC) 協定的外部身分識別提供者。
VKS 在 Supervisor 和 VKS 叢集節點上執行 Pinniped OIDC 用戶端。為 Supervisor 設定外部 OIDC 提供者會自動設定 Pinniped 元件。
為了與 Supervisor 進行驗證，VKS 支援 VCF CLI。

5.2. Container Storage Interface (CSI)

paravirtual CSI plugin 是一個在 VKS 叢集內執行的 Kubernetes Pod，它透過 Supervisor 與 VMware Cloud Native Storage (CNS) 整合。在 VKS 叢集中執行的 Kubernetes Pod 可以掛載三種類型的虛擬磁碟：臨時磁碟 (ephemeral)、永久性磁碟區 (persistent volume) 和容器映像檔 (container image)。
- 暫態儲存 (Transient Storage) Pod 需要暫態儲存來存放 Kubernetes 物件等臨時性資料，例如日誌、磁碟區和組態對應 (configuration maps)。暫態儲存的生命週期與 Pod 共存。臨時性資料在容器重啟後仍然存在，但一旦 Pod 被刪除，儲存臨時性資料的虛擬磁碟也會隨之消失。
- 永久性儲存 (Persistent Storage) VKS 利用 vSphere 儲存策略框架來定義儲存類別 (storage classes) 和保留永久性磁碟區。VKS 叢集會將永久性磁碟區的請求傳遞給與 VMware Cloud Native Storage (CNS) 整合的 Supervisor。永久性磁碟區可以使用儲存類別在叢集節點上動態置備，也可以手動置備。
- 貨櫃映像檔儲存 (Container Image Storage) Kubernetes Pod 內的 Containers 會使用包含待執行軟體的 image。Pod 會將其 containers 所使用的映像檔掛載為映像檔虛擬磁碟。當 Pod 完成其生命週期時，映像檔虛擬磁碟會從 Pod 卸離。Kubelet 負責從 image registry 中提取 container image，並將其轉換為可在 Pod 內執行的虛擬磁碟。

5.3. Container Network Interface (CNI)

Container Network Interface plugin 是一種提供 Pod 網路功能的 CNI plugin。
VKS 叢集支援以下 Container Network Interface (CNI) 選項：Antrea (預設) 和 Calico。此外，VKS 還提供 Antrea NSX Routed CNI 來實作可路由的 Pod 網路。
下表總結了 VKS 叢集的網路功能及其實作方式：

5.4. Cloud Provider Implementation

Cloud Provider Implementation 讓你建立 Kubernetes load balancer 和 ingress services.

6. VKS Cluster API

VKS 提供一個用於配置 (provisioning) 和管理 VKS clusters 生命週期的 API。v1beta1 API 可讓您根據 VMware 提供的預設 ClusterClass，建立符合規範的 Kubernetes 叢集 (類型為 Cluster)。

7. VKS Cluster Clients

Supervisor 上的 VKS 支援多種用戶端介面，用於設定、監控和管理 VKS 叢集。

vSphere Client：用於設定 Supervisor 並監控已部署的 VKS 叢集。
VCF CLI：用於透過 vCenter Single Sign-On 向 Supervisor 和 VKS 叢集進行驗證。
kubectl：用於以宣告方式 (declaratively) 設定和管理 VKS 叢集的生命週期，並與 Supervisor 互動。
Tanzu Mission Control 網頁介面：用於管理 VKS 叢集。

Node Restoration at the VM Level Troubleshooting

Sun, 29 Jun 2025 08:48:06 +0800

問題背景

在 Proxmox UI 對 VM Take Snapshot(Include RAM)，過幾天後，把 VM Restore 回去。

K8s 環境資訊

NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
tkbp-control-plane Ready control-plane 2d22h v1.32.5 172.22.8.1 <none> Debian GNU/Linux 12 (bookworm) 6.8.0-62-generic containerd://2.1.1
tkbp-worker1 Ready taroko-worker 2d22h v1.32.5 172.22.8.2 <none> Debian GNU/Linux 12 (bookworm) 6.8.0-62-generic containerd://2.1.1
tkbp-worker2 Ready taroko-gateway 2d22h v1.32.5 172.22.8.3 <none> Debian GNU/Linux 12 (bookworm) 6.8.0-62-generic containerd://2.1.1

CNI 使用 Canal 版本 v3.29.2，flannel 版本 flannel:v0.24.4，設定資訊

問題描述

當使用帶有 Canal CNI 的 Kubeadm 叢集時，Pod 卡在 ContainerCreating 無法啟動，如下 :

$ kubectl get pods -n user9
NAME READY STATUS RESTARTS AGE
n1 0/1 ContainerCreating 0 141m

去 Describe pod 看到的 Event 資訊如下

 Warning FailedCreatePodSandBox 4m52s (x604 over 137m) kubelet (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "25c71fb49bcd00ce537246d4484b8eedba5277e9504163ec1c18f420d0d6f190": plugin type="calico" failed (add): error getting ClusterInformation: connection is unauthorized: Unauthorized

解決方案

kubectl -n kube-system rollout restart ds canal

執行以下命令，二次確認 Canal Pods 皆重啟

# kubectl -n kube-system get pods -l k8s-app=canal
NAME READY STATUS RESTARTS AGE
canal-4xkb9 2/2 Running 0 52s
canal-b2brv 2/2 Running 0 85s
canal-xqqcm 2/2 Running 0 68s

Root Cause

TL;DR: Canal Service Account 的 Token 過期

Calico 的 calico-kube-controllers pod 時無法在預定時間內從 Kubernetes API Server 取得 ClusterInformation 資源，導致初始化失敗

$ kubectl -n kube-system logs calico-kube-controllers-6b65fb5f89-77695

看到以下錯誤訊息

...
[ERROR][1] kube-controllers/client.go 320: Error getting cluster information config ClusterInformation="default" error=Get "https://10.98.8.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default": context deadline exceeded
[INFO][1] kube-controllers/client.go 248: Unable to initialize ClusterInformation error=Get "https://10.98.8.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default": context deadline exceeded
[INFO][1] kube-controllers/client.go 254: Unable to initialize default Tier error=client rate limiter Wait returned an error: context deadline exceeded
[INFO][1] kube-controllers/client.go 260: Unable to initialize adminnetworkpolicy Tier error=client rate limiter Wait returned an error: context deadline exceeded
[ERROR][1] kube-controllers/main.go 256: Failed to verify datastore error=Get "https://10.98.8.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default": context deadline exceeded
[INFO][1] kube-controllers/main.go 290: Health check is not ready, retrying in 2 seconds with new timeout: 8s
[INFO][1] kube-controllers/resources.go 378: Terminating main client watcher loop
[INFO][1] kube-controllers/resources.go 350: Main client watcher loop
...

測試使用 calico 的 kubeconfig 連線 API Server

kubectl --kubeconfig /etc/cni/net.d/calico-kubeconfig auth whoami

執行結果

error: You must be logged in to the server (Unauthorized)

由以上結果得知問題點在 calico 的 kubeconfig

執行以下命令，確認 kubeconfig 中 calico user (在 K8s 中是 Service Account) 的 Token 有效期限

yq e '.users[0].user."token"' /etc/cni/net.d/calico-kubeconfig | \
tr -d '\n' | \
cut -d '.' -f 2 | \
base64 -d | \
jq

yq 安裝命令 : wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/local/bin/yq && chmod +x /usr/local/bin/yq

執行結果 :

{
 "aud": [
 "https://kubernetes.default.svc.tkbp.k8s"
 ],
 "exp": 1750924118,
 "iat": 1750837718,
 "iss": "https://kubernetes.default.svc.tkbp.k8s",
 "jti": "378d4f97-c792-4a28-a547-379307b047d7",
 "kubernetes.io": {
 "namespace": "kube-system",
 "serviceaccount": {
 "name": "canal",
 "uid": "a8e73c8a-8064-46a8-bd6c-f2badbddf663"
 }
 },
 "nbf": 1750837718,
 "sub": "system:serviceaccount:kube-system:canal"
}

iat：token 的發行時間。 exp：token 的過期時間。

將 Token 的時間轉換為人類可閱讀的格式

# date -d @1750924118

執行結果 :

Thu Jun 26 07:48:38 UTC 2025

成功找到問題點，機器當前的時間已在 6/28，Token 已過期

查找 Token 發行日期

# date -d @1750837718
Wed Jun 25 07:48:38 UTC 2025

由以上結果得知 Service Account 的 Token 預設有效日期為 1 天

exec 進 pod 檢查 SA Token 的命令

cat /var/run/secrets/kubernetes.io/serviceaccount/token \
| cut -d "." -f2 \
| tr -d "\n" \
| awk '{pad=4-(length($0)%4); if(pad<4) printf "%s", $0 sprintf("%"pad"s","")}' \
| tr ' ' '=' \
| base64 -d \
| jq

執行結果 :

{
"aud": [
"https://kubernetes.default.svc.tkbp.k8s"
],
"exp": 1782720071,
"iat": 1751184071,
"iss": "https://kubernetes.default.svc.tkbp.k8s",
"jti": "e64feb22-412e-4065-b9b9-5c45889958a9",
"kubernetes.io": {
"namespace": "taroko",
"node": {
"name": "tkbp-worker2",
"uid": "f6595a85-3a61-411b-a186-1c2f6bd32a26"
},
"pod": {
"name": "taroko-tkadm-794b86456f-qbwlf",
"uid": "e4fd32f5-a9c0-41d1-aa34-b63203ccd3d0"
},
"serviceaccount": {
"name": "tkadm",
"uid": "21b873fc-1ad0-4442-a66e-acf5fbe78311"
},
"warnafter": 1751187678
},
"nbf": 1751184071,
"sub": "system:serviceaccount:taroko:tkadm"
}

VMware Cloud Foundation 5.2 Architect

Sat, 28 Jun 2025 06:27:50 +0800

Question 1

During a requirements gathering workshop, several Business and Technical requirements were captured from the customer. Which requirement is classified as a Technical Requirement?

在需求蒐集工作坊期間，已從客戶收集了數個業務需求與技術需求。哪一項需求屬於技術需求？

A. Reduce system processing time for service requests by 25%. B. The system must support 5,000 concurrent users. C. Increase customer satisfaction by 15%. D. Expand market reach to include new geographical regions.

Here’s why: 在 VMware Cloud Foundation（VCF）架構中，需求會根據其焦點被分類為業務需求或技術需求。技術需求會具體說明可衡量的系統能力或限制，並直接影響計算、儲存或網路等基礎架構元件的設計決策。相對地，業務需求則著重於 IT 所要支援的組織目標或成果。

選項 B「系統必須支援 5,000 名同時用戶」就是技術需求，因為它定義了一個具體的系統容量指標（同時用戶數），會直接影響 VCF 設計中的可伸縮性和資源配置，例如決定工作負載域（workload domains）的規模或 NSX 設定。

選項 A「將服務請求的系統處理時間減少 25%」雖然可能包含技術性，但通常是源自業務目標（效率）的衍生需求，因此在此情境中並不那麼明確屬於技術需求。

選項 C 和 D 則專注於客戶滿意度和市場覆蓋，顯然屬於業務導向，與組織成果相關，而非系統規格。參考資料：VMware Cloud Foundation 5.2 Architect Study Guide，第2章：需求蒐集與分析，之「需求分類」一節。

Question 2

During a requirement gathering workshop, various Business and Technical requirements were collected from the customer. Which requirement would be categorized as a Business Requirement?

在需求蒐集工作坊期間，從客戶那裡收集了各種業務及技術需求。哪一項需求會被歸類為業務需求？

A. The application should be compatible with Windows, macOS, and Linux operating systems. B. Decrease processing time for service requests by 30%. C. The system should support 10,000 concurrent users. D. Data should be encrypted using AES-256 encryption.

Here’s why: 在 VCF 中，業務需求說明了解決方案必須實現的組織目標，通常聚焦於效率、成本或服務改進，而不是具體的技術實作。

選項 B：「將服務請求的處理時間減少 30%」是業務需求，因為它針對一個能改善客戶服務交付的操作效率目標，以業務的角度可衡量，而不說明系統如何達成此目標。

選項 A、C 和 D——指定作業系統相容性、用戶容量與加密標準——則是技術需求，因為它們詳細說明了架構師必須在 VCF 元件（如 vSphere 或 NSX）中實作的系統能力或安全機制。區別關鍵在於意圖：B 著重在成果（速度），而其它則定義系統屬性。

參考：VMware Cloud Foundation 5.2 Architect Study Guide，第2章：需求分類，之「業務 vs. 技術需求」一節。

Question 3

An organization is planning to expand their existing VMware Cloud Foundation (VCF) environment to meet an increased demand for new user-facing applications. The physical host hardware proposed for the expansion is a different model compared to the existing hosts, although it has been confirmed that both sets of hardware are compatible.The expansion needs to provide capacity for management tooling workloads dedicated to the applications, and it has been decided to deploy a new cluster within the management domain to host the workloads. What should the architect include within the logical design for this design decision?

某組織計畫擴充其現有的 VMware Cloud Foundation (VCF) 環境，以因應對新用戶面向應用程式日益增加的需求。提議用於擴充的實體主機硬體與現有主機採用不同型號，但已證實這兩批硬體相互相容。此擴充需為專屬於這些應用程式的管理工具工作負載提供容量，並決定在管理域內部署一個新叢集以承載這些工作負載。在邏輯設計階段，架構師應該包含哪些內容以說明此設計決策？

A. The design justification stating that the separate cluster provides flexibility for manageability and connectivity of the workloads B. The design assumption stating that the separate cluster will provide complete isolation for lifecycle management C. The design implication stating that the management tooling and the VCF management workloads have different purposes D. The design qualities affected by the decision listed as Availability and Performance

Here’s why: 在 VCF 中，邏輯設計會記錄設計決策如何與需求對齊，通常透過合理性說明、假設或影響分析來呈現。在此情境中，為了專門承載管理工具工作負載而在管理域內新增一個叢集，需要在邏輯設計中提供其合理性。

選項 A 所提供的合理性──「這個獨立叢集可提升可管理性與連通性的靈活性」──與 VCF 的工作負載隔離及運營效率原則相符。它說明了為何做出此決策──提升管理工具的運作彈性──而並未假設情境中並未支持的結果（如選項 B 所提「完全隔離」）或僅列出效果（如選項 C 與 D）。

在 VCF 5.2 的架構中，管理域確實可以用來承載額外的叢集以達成此類目的，而此合理性正直接對應到為管理工具工作負載提供專屬容量的需求。

參考資料：VMware Cloud Foundation 5.2 Planning and Preparation Guide，第4章：邏輯設計考量，之「設計合理性說明」一節。

Question 4

An architect is designing a VMware Cloud Foundation (VCF)-based private cloud solution for a customer. The customer has stated the following requirement:

All management tooling must be resilient against a single ESXi host failure

When considering the design decisions for VMware Aria Suite components, what should the Architect document to support the stated requirement?

一位架構師正在為客戶設計基於 VMware Cloud Foundation (VCF) 的私有雲解決方案。客戶提出以下需求：

所有管理工具必須能夠在單一 ESXi 主機故障時保持彈性

在考量 VMware Aria Suite 元件的設計決策時，架構師應記錄哪些內容來支持此需求？

A. The solution will deploy the VCF Workload domain in a stretched topology across two sites. B. The solution will deploy three Aria Automation appliances in a clustered topology. C. The solution will deploy Aria Suite Lifecycle in a clustered topology. D. The solution will deploy an external load balancer for Aria Operations Cloud Proxies.

Here’s why: 要針對單一 ESXi 主機故障提供韌性，需在 VCF 中為管理元件實作高可用性（HA）。VMware Aria Suite（包括 Aria Automation）透過叢集架構支援 HA。

選項 B：「在叢集拓撲中部署三部 Aria Automation 裝置」可確保當其中一部主機故障時，其餘兩部仍能維持服務，直接滿足需求。對於 Aria Automation 而言，三節點叢集是實現 HA 的最小配置，可在 VCF 管理域內提供容錯能力。

選項 A（延伸式工作負載域）與管理工具的 HA 無關；

選項 C（Aria Suite Lifecycle 叢集化）並非該元件的標準 HA 功能；

選項 D（為 Operations 代理設置負載平衡器）則是針對不同元件與目的。

Question 5

A customer has a requirement to improve bandwidth and reliability for traffic that is routed through the NSX Edges in VMware Cloud Foundation. What should the architect recommend satisfying this requirement?

A. Configure a Load balanced Group for NSX Edges B. Configure a TEP Group for NSX Edges C. Configure a TEP Independent Group for NSX Edges D. Configure a LAG Group for NSX Edges

Here’s why:

Question 6

A VMware Cloud Foundation multi-AZ (Availability Zone) design mandates that:

All management components are centralized.
The availability SLA must adhere to no less than 99.99%.

What would be the two design decisions that would help satisfy those requirements? (Choose two.)

A. Choose two distant AZs and configure distinct management workload domains. B. Configure a stretched L2 VLAN for the infrastructure management components between the AZs. C. Configure a separate VLAN for the infrastructure management components within each AZ. D. Configure VMware Live Recovery between the selected AZs. E. Choose two close proximity AZs and configure a stretched management workload domain.

B, E

Here’s why:

Question 7

A customer has a requirement to use isolated domains in VMware Cloud Foundation but is constrained to a single NSX management pane. What should the architect recommend satisfying this requirement?

A. An NSX VPC B. A Shared NSX Instance C. NSX Federation D. A 1:1 NSX Instance

Here’s why:

Question 8

An Architect is designing a VMware Cloud Foundation (VCF)-based private cloud solution for a customer. During the requirements gathering workshop, the customer stated the following:

All users must only have access to the solution components to fulfill their defined role.
All administrative users must be authenticated to a separate approved identity source for administrator accounts only.
All service users must be authenticated to the central approved identity source.
All service account passwords must be stored centrally in an approved secrets management platform.

When creating the design, how should the Architect classify all the stated requirements?

A. Security B. Manageability C. Recoverability D. Availability

Here’s why:

Question 9

During a transformation project kick-off meeting, an architect highlights specific areas on which to focus while developing the new conceptual design. Which statement is the business requirement?

A. The solution must continue to operate even in case of an entire datacenter failure. B. The project should use the existing storage devices within the data center. C. Sites must support a network latency of less than 12 ms RTT. D. There is no budget specifically assigned for disaster recovery.

Here’s why:

Question 10

wing requirements were identified in an architecture workshop for a virtual infrastructure design project.

REQ001: All virtual machines must satisfy the Recovery Point Objective (RPO) of fifteen (15) minutes or less in a disaster recovery (DR) situation
REQ002: Service level availability must satisfy 99.999% measured yearly.

Which two test cases will validate these requirements?

A. Simulate or invoke an outage of the primary datacenter. All virtual machines must be restored within fifteen (15) minutes or less. B. Simulate or invoke an outage of the primary datacenter. All virtual machines must not lose more than one (1) hour of data prior to the outage. C. Simulate or invoke an outage of the primary datacenter. All virtual machines must not lose more than fifteen (15) minutes of data prior to the outage. D. Simulate or invoke an outage of the primary datacenter. All virtual machines must be restored within one (1) hour or less.

A, C

單選題答案 C

Here’s why:

Question 11

As part of a new VMware Cloud Foundation (VCF) deployment, a customer is planning to implement vSphere IaaS control plane. What component could be installed and enabled to implement the solution?

A. Aria Automation B. NSX Edge networking C. Storage DRS D. Aria Operations

Here’s why:

Question 12

An architect is preparing a VI Workload Domain design with a dedicated NSX instance. The workload domain is planned to grow up to 300 ESXi hosts within the next six months. Which is the minimum NSX Manager form factor that should be recommended by the architect for this VI Workload Domain to support the forecasted growth?

A. Large B. Medium C. Extra Small D. Small

Here’s why:

Question 13

A customer is deploying VCF at a new datacenter location. They will migrate their workloads from the existing datacenter to the new VCF platform over six months. Both datacenters will run simultaneously for six months during the migration. Which of the following should be a documented risk?

A. Six months may not be enough time to complete the migration. B. There will be connectivity between the two locations. C. Bandwidth between the two locations is sufficient to accommodate the workload migration. D. Workloads will be powered off during migration.

Here’s why:

Question 14

An architect had gathered the following requirements and constraints for a VMware Cloud Foundation (VCF) deployment. Requirements:

User interface (UI) SSL certificates must have a maximum validity of 6 months.
Have the least possible administrative time to install and renew certificates.
Each certificate must be created on a per VCF component basis. Constraints:
Limited administrative skillsets on SSL certificate administration
Limited operational expenditure budget for SSL certificates

Which design decision should be made to satisfy the stated requirement(s) and constraint(s)?

A. Use wildcard certificates B. Use and configure integration with a certificate vendor such as DigiCert C. Disable the use of SSL certificates for user interfaces D. Use and configure integration with Microsoft Certificate Authority (CA)

Here’s why:

Question 15

A design requirement has been specified for a new VMware Cloud Foundation (VCF) instance. All managed workload resources must be lifecycle managed with the following criteria:

Development resources must be automatically reclaimed after two weeks
Production resources will be reviewed yearly for reclamation
Resources identified for reclamation must allow time for review and possible extension

What capability will satisfy the requirements?

A. Aria Suite Lifecycle Content Management B. Aria Operations Rightsizing Recommendations C. Aria Automation Lease Policy D. Aria Automation Project Membership

Here’s why:

Question 16

A VMware Cloud Foundation design incorporates the following technical requirements:

All management components must have their login sessions timeout after 2 minutes of inactivity.
Communication between management components should be limited to required ports only.
Modifications required by compliancy should not impact the management components’ functionality.

What would be the recommendation from a design perspective that would aid in achieving the above requirements?

A. Consult the vSphere Security Configuration kit B. Leverage the results of a vulnerability assessment and apply the recommendations C. Consult the Compliance Kit for VMware Cloud Foundation D. Apply NSX DFW (Distributed Firewall) to achieve zero-trust

Here’s why:

Question 17

An architect is designing a new VMware Cloud Foundation (VCF)-based Private Cloud solution. During the requirements gathering workshop, a network team stakeholder stated that:

The solution must ensure that any physical networking component has N + N redundancy.
The solution must ensure that inter-datacenter network links are diversely routed.

When documenting the design, how should the architect classify these requirements?

A. Recoverability B. Availability C. Performance D. Manageability

Here’s why:

Question 18

Which Operating System (OS) is not supported by Aria Operations for OS and Application Monitoring?

A. Windows Server 2012 R2 B. CentOS C. Windows Server 2012 D. MacOS

Here’s why:

Question 19

An architect has been tasked with reviewing a VMware Cloud Foundation design document. Observe the following requirements:

REQ01: The solution must provide the ability to request new tenant creation with multi-site and different size options.
REQ02: The solution must provide the capability to monitor the software-defined data center for capacity and performance.
REQ03: The solution must provide the ability to generate reports with customized metrics to meet business requests.
REQ04: The solution should report all capacity planning components (such as current capacity usage monthly and annual usage growth).
REQ05: The solution must provide the ability to provision new virtual machines from predefined templates.
REQ06: The solution must provide a self-service catalog for end-users to consume services.

Observe the following design decisions:

DD01: There will be a centralized deployment of Aria Operations Management.
DD02: There will be customized super-metrics based on existing metrics.

Based on the stated requirements and design decisions, which three requirements does this design decision satisfy? (Choose three.)

A. REQ05 B. REQ01 C. REQ06 D. REQ04 E. REQ03 F. REQ02

D, E, F

Here’s why:

Question 20

An architect has come up with a list of design decisions after a workshop with the business stakeholders. Which design decision describes a logical design decision?

A. Asynchronous storage replication that satisfies a recovery point objective (RPO) of 15min betweensite A and B B. Both sites A and B will have a /16 dedicated network subnets. C. End users will interact with application server hosted in Site A D. End users should always experience instantaneous application response

Here’s why:

Question 21

A customer defined a requirement for the newly deployed SDDC infrastructure which will host one of the applications responsible for video streaming. Application will run as part of a VI Workload Domain with dedicated NSX instance and virtual machines. Required network throughput was defined as 250 Gb/s. Additionally, the application should provide the lowest possible latency. Which design decision should be recommended by an architect for the NSX Edge deployment?

A. Deploy 2 NSX Edges using NSX console and add to Edge cluster created in SDDC Manager. B. Deploy 4 extra large edges using vCenter Server console. C. Deploy NSX bare-metal Edges and create Edge Cluster using NSX console. D. Deploy 2 large NSX Edges using SDDC Manager.

Here’s why:

Question 22

A company will be expanding their existing VCF environment for a new application. The existing VCF environment currently has a management domain and two separate VI workload domains with different hardware profiles. The new application has the following requirements:

The application will use significantly more memory than current workloads today.
The application will have a limited number of licenses to run on hosts.
Additional VCF and hardware costs have been approved for the application.
The application will contain confidential customer information that requires isolation from other workloads.

What design recommendation should the administrator document?

A. Deploy a new consolidated VCF instance and deploy the new application into it. B. A new Workload domain with hardware supporting the memory requirements of the new application should be implemented. C. Enough identical hardware for the management domain should be ordered to accommodate the new application requirements and a new workload domain should be designed for the application. D. Purchase enough matching hardware to accommodate the new application’s memory requirements and expand an existing cluster to accommodate the new application. Use host affinity rules to manage the new licensing.

Here’s why:

Question 23

An architect is working with an organization on the creation of a new Private Cloud Platform. The organization has provided the following business objectives they wish to achieve with the new platform:

Reduce the operating costs associated with running separate areas of hosting capacity and separate/duplicate systems.
Reduce the risks, time, and effort associated with managing platforms that are out of vendor support.
Reduce the operating costs associated with Public Cloud usage.
Reduce the risks associated with having incomplete documentation for application inventory and dependency mappings. They have grouped these business objectives into a set of use cases:
Migration - Provide a platform that supports the migration of virtualized workloads from existing platforms.
Containerization - Provide a platform that supports the deployment of containerized workloads.
Centralization and Consolidation - Provide a central private cloud platform accessible to all relevant areas of the business.

When considering these objectives and use cases, what should the architect include in the design documentation as a part of the Conceptual Model?

A. An assumption that the new platform will co-exist with the existing platforms for a period of time to allow workloads to be migrated in a phased approach B. A risk that the existing platforms are running Linux Operating Systems that are out of vendor support C. An assumption that a complete mapping of application dependencies is not available D. A requirement that the solution will provide the capability to migrate Kubernetes-based workloads from the Public Cloud

Here’s why:

Question 24

A customer has stated the following requirements for Aria Automation within their VCF implementation:

Users must have access to specific resources based on their company organization
Developers must only be able to provision to the Development environment
Production workloads can be placed on DMZ or Production clusters

What two design decisions must be implemented to satisfy these requirements? (Choose two.)

A. Separate cloud zones will be configured for Development and Production. B. Users’ access to resources will be controlled by project membership. C. Users’ access to resources will be controlled by tenant membership. D. Separate tenants will be configured for Development and Production.

A, B

Here’s why:

Question 25

The following design decisions were made relating to storage design:

A storage policy that would support failure of a single fault domain being the server rack
Two vSAN OSA disk groups per host each consisting of four 4TB Samsung SSD capacity drives
Two vSAN OSA disk groups per host each consisting of a single 300GB Intel NVMe cache drive
Encryption at rest capable disk drives
Dual 10Gb or faster storage network adapters

Which two design decisions would an architect include within the physical design? (Choose two.)

A. A storage policy that would support failure of a single fault domain being the server rack B. Two vSAN OSA disk groups per host each consisting of a single 300GB Intel NVMe cache drive C. Encryption at rest capable disk drives D. Dual 10Gb or faster storage network adapters E. Two vSAN OSA disk groups per host each consisting of four 4TB Samsung SSD capacity drives

B, E

Here’s why:

Question 26

The following requirements were identified in an architecture workshop for a virtual infrastructure design project.

REQ001: All virtual machines must meet the Recovery Time Objective (RTO) of twenty-four hours or less in a disaster recovery (DR) scenario.

Which two test cases will verify these requirements?

A. Simulate or trigger an outage of the primary datacenter. All virtual machines must be restored within four hours or less. B. Simulate or trigger an outage of the primary datacenter. All virtual machines must be restored within twenty-four hours or less. C. Simulate or trigger an outage of the primary datacenter. All virtual machines must not lose more than twenty-four hours of data prior to the outage. D. Simulate or trigger an outage of the primary datacenter. All virtual machines must not lose more than four hours of data prior to the outage.

單選答案 B B, C

Here’s why:

Question 27

During a design discussion, the VMware Cloud Foundation Architect was presented with a requirement to reduce power utilization across all workload domains including management. The architect has suggested to use vSphere Distributed Power Management (DPM) to satisfy this requirement. Which recommendation should the architect provide?

A. vSphere DPM for Management Workload Domain (excluding when vSAN is a principal storage). B. vSphere DPM for VI Workload Domains (excluding when vSAN is a principal storage). C. vSphere DPM for Management Workload Domain (only when hosted within a Hyperscaler Data Center). D. vSphere DPM for VI Workload Domains (any principal storage). E. vSphere DPM for Management Workload Domain (any principal storage).

Here’s why:

Question 28

During the requirements gathering workshop for a new VMware Cloud Foundation (VCF)-based Private Cloud solution, the customer states that the solution must:

Provide sufficient capacity to migrate and run their existing workloads.
Provide sufficient initial capacity to support a forecasted resource growth of 30% over the next 3 years.

When creating the design document, under which design quality should the architect classify these stated requirements?

A. Availability B. Performance C. Recoverability D. Manageability

Here’s why:

Question 29

During the requirements gathering workshop for a new VMware Cloud Foundation (VCF)-based Private Cloud solution, the customer states that the solution must:

Provide a single interface for monitoring all components of the solution.
Minimize the effort required to maintain the solution to N-1 software versions.

When creating the design document, under which design quality should the architect classify these stated requirements?

A. Manageability B. Recoverability C. Availability D. Performance

Here’s why:

Question 30

An architect is planning resources for a new cluster that will be integrated into an existing VI Workload Domain. The cluster’s primary purpose is to support a mission-critical application with five resource-intensive virtual machines. Which design recommendation should the architect provide to prevent resource bottlenecks while meeting the N+1 availability requirement and keeping the overall investment cost minimal?

A. Establish a cluster with four hosts and implement rules to prioritize resources for the application virtual machines. B. Establish a cluster with three hosts and exclusively run the application virtual machines on this cluster. C. Establish a cluster with six hosts and implement automated placement rules to keep the application virtual machines together. D. Establish a cluster with six hosts and implement automated placement rules to distribute the application virtual machines.

Here’s why:

Question 31

An architect has been asked to recommend a solution for a mission-critical application running on a single virtual machine to ensure consistent performance. The virtual machine operates within a vSphere cluster of four ESXi hosts, sharing resources with other production virtual machines. There is no additional capacity available. What should the architect recommend?

A. Use CPU and memory reservations for the mission-critical virtual machine. B. Use CPU and memory limits for the mission-critical virtual machine. C. Create a new vSphere Cluster and migrate the mission-critical virtual machine to it. D. Add additional ESXi hosts to the current cluster.

Here’s why:

Question 32

An architect is designing a VMware Cloud Foundation (VCF)-based private cloud solution for a customer. The customer has stated the following requirement:

All components within the solution must be resilient to N+1.

During discovery, the following information has also been provided:

Over the next 3 years, due to various applications being retired, no overall growth in resource consumption is expected.

Following a review of a demand-based capacity report from Aria Operations, the architect has calculated that all of the existing workloads should fit into a 4-node cluster. Once all workloads are migrated, the resources of the cluster will be 90% utilized.

Given the information provided, a combination of which three design decisions satisfy the requirement? (Choose three.)

A. The solution will set the DRS Automation level setting for the workload cluster to Partially Automated. B. The solution will deploy a workload cluster consisting of five VMware vSphere hosts. C. The solution will set the Host failures cluster tolerates for the workload cluster to 1. D. The solution will deploy a workload cluster consisting of four VMware vSphere hosts. E. The solution will configure vSphere High Availability (HA) for the workload cluster. F. The solution will configure vSphere Dynamic Resource Scheduling (DRS) for the workload cluster.

B, C, E

Here’s why:

Question 33

An architect is updating a design document in preparation for an expansion of their organization’s existing VCF environment. Following the completion of a capacity assessment, a new cluster will be deployed to support the hosting of future application deployments. Due to restrictions on the availability of budget for the project, the hardware for the additional cluster has already been procured and there is no additional budget available for future procurements. What should the architect include within the design documentation based on this approach?

A. A constraint that the procured hardware must be used due to budget restrictions. B. A risk that additional hardware is not available for purchase. C. A requirement that the cluster must be deployed within the existing workload domain. D. An assumption that the new cluster will provide sufficient capacity for the applications.

Here’s why:

Question 34

An architect decided to deploy an NSX Edge cluster using SDDC Manager. These Edges will be used by a Tier-0 Gateway configured with BGP to provide North-South connectivity in the Management Domain. Which statement justifies this design decision?

A. NSX Edges deployed via SDDC Manager can be updated separately in the future. B. VPN service in NSX will be available and configurable via SDDC Manager with NSX Edges deployed using this method. C. Extra Large form factor is available only when edges are deployed using SDDC Manager. D. This deployment method will automatically configure dynamic routing.

Here’s why:

Question 35

An architect is working on a design for a new VMware Cloud Foundation (VCF) solution for a retail organization. The organization wants to initially deploy the solution into their headquarters and a number of larger stores. They also plan to pilot the expansion of the deployment into some of their smaller stores. The locations have the following characteristics:

Headquarters has a brand-new datacenter and 40Gb network infrastructure.
Larger stores have secure machine rooms and 10Gb network infrastructure.
Smaller stores have small secure racks and 100Mb network infrastructure.

The organization’s cloud administration team have stated a requirement that the design should minimize the number of instances of management tools they need to support without impacting the performance of the workloads consumed by the end users. What three design decisions about the VCF deployment architecture could the architect include in the logical design? (Choose three.)

A. Headquarters will have a private cloud based on the VCF Consolidated Architecture. B. Larger stores will have a private cloud based on the VCF Consolidated Architecture. C. Smaller stores will have remote clusters deployed from the HQ VCF instance. D. Smaller stores will have remote clusters deployed from the geographically closest Larger store VCF instance. E. Headquarters will have a private cloud based on the VCF Standard Architecture. F. Larger stores will have workload domains deployed from the HQ VCF instance.

C, E, F

Here’s why:

Question 36

An architect is responsible for designing a new VMware Cloud Foundation environment and has identified the following requirements provided by the customer:

REQ01: The database server must support a minimum of 15,000 transactions per second.
REQ02: The design must satisfy PCI-DSS compliance.
REQ03: The storage network must have a minimum latency of 10 milliseconds prior to path failover.
REQ04: The Production environment must be deployed into the primary data center.
REQ05: The platform must be capable of running 1500 virtual machines across both data centers.

What are the two functional requirements? (Choose two.)

A. The design must satisfy PCI-DSS compliance. B. The database server must support a minimum of 15,000 transactions per second. C. The storage network must have a minimum latency of 10 milliseconds prior to path failover. D. The Production environment must be deployed into the primary data center. E. The platform must be capable of running 1500 virtual machines across both data centers.

B, E

Here’s why:

Question 37

During a requirements gathering workshop, several Business and Technical requirements were captured from the customer. Which requirement will be classified as a Business Requirement?

A. Reduce processing time for service requests by 30%. B. The system must support 10,000 concurrent users. C. Data must be encrypted using AES-256 encryption. D. The application must be compatible with Windows, macOS, and Linux operating systems.

Here’s why:

Question 38

An architect is designing a VMware Cloud Foundation (VCF)-based private cloud solution for a customer that will include two physical locations. The customer has stated the following requirement: All management tooling must be resilient at the component level within a single site. When considering the design decisions for VMware Aria Suite components, what should the Architect document to meet the stated requirement?

A. The solution will implement an external load balancer for Aria Operations Cloud Proxies. B. The solution will configure the VCF Workload domain in a stretched topology across two locations. C. The solution will deploy three Aria Automation appliances in a clustered configuration. D. The solution will deploy Aria Suite Lifecycle Manager in a high availability configuration.

Here’s why:

Question 39

An architect is designing a VMware Cloud Foundation (VCF)-based Private Cloud solution. During the requirements gathering workshop with the customer stakeholders, the following information was noted:

In the event of a site-level disaster, the solution must enable all production workloads to be restarted in the secondary site.

In the event of a host failure, workloads must be restarted in priority order.

When creating the design documentation, which design quality should be used to classify the stated requirements?

A. Availability B. Manageability C. Performance D. Recoverability

Here’s why:

Question 40

An Architect is responsible for designing a VMware Cloud Foundation (VCF)-based solution for a customer. During the discovery workshop, the following requirements were stated by the customer:

All applications/workloads designated as business critical have a Recovery Point Objective (RPO) of 1 business hour.

The infrastructure components of the VCF solution must have a Recovery Time Objective (RTO) of 4 business hours.

In the context provided, what does the RTO measure?

A. It determines the minimum amount of data loss that can be tolerated. B. It determines the maximum tolerable amount of time allowed before an application/service should be recovered to a usable state. C. It determines the minimum tolerable amount of time allowed before an application/service should be recovered to a usable state. D. It determines the maximum amount of data loss that can be tolerated.

Here’s why:

Question 41

The following requirements were identified in an architecture workshop for a VMware Cloud Foundation (VCF) design project utilizing vSAN for its primary storage solution:

REQ001: Application must maintain a minimum of 1,000 transactions per second (TPS) during business hours excluding disaster recovery (DR) scenarios.
REQ002: Automatic DRS and HA must be utilized.
REQ003: Planned maintenance must be executed outside of business hours.

Which of the following test scenarios should be added and performed to validate these requirements?

A. Trigger a Virtual Machine vMotion operation. B. Trigger a vCenter Server update. C. Trigger a vSAN disk group evacuation. D. Trigger a failure of an ESXi host.

Here’s why:

Question 42

An Architect has been tasked with reviewing a VMware Cloud Foundation design document. Observe the following requirements:

REQ01: The solution must support the private cloud cybersecurity industry and local standards and controls.
REQ02: The solution must ensure that the cloud services are transitioned to operation teams.
REQ03: The solution must provide a self-service portal.
REQ04: The solution must provide the ability to consume storage based on policies.
REQ05: The solution should provide the ability to extend networks between different availability zones.
REQ06: The solution should allow only supported versions of management solutions to be deployed.

Observe the following design decisions:

DD01: There will be a clustered deployment of Aria Automation.
DD02: There will be an integration between Aria Automation and multiple geo-located vCenter Servers.
DD03: Aria Suite Lifecycle will be deployed to provide lifecycle management of Aria Suite components.

Based on the stated requirements, what are the three implications for taking the stated design decisions? (Choose three.)

A. Aria Automation must have network access to all vCenter Servers. B. Aria Suite Lifecycle should be deployed through the SDDC Manager. C. An external database is required for Aria Automation clustering. D. A load balancer is required for Aria Automation high availability. E. The latency between the Aria Automation Appliances must be less than 2ms. F. The vCenter Servers must have network access to each other.

A, D, E

Here’s why:

Question 43

An architect is evaluating a requirement for a Cloud Management self-service solution to offer its users the ability to migrate their own workloads using VMware vMotion. Which component could the architect include in the solution design that will help satisfy the requirement?

A. Aria Suite Lifecycle Manager B. Aria Automation Orchestrator C. Aria Operations D. Aria Automation Config

Here’s why:

Question 44

An architect is designing a VMware Cloud Foundation (VCF)-based private cloud solution for a customer. During the requirements gathering workshop, the customer provided the following requirement:

All SSL certificates should be provided by the company’s certificate authority.

When creating the design, how should the architect classify this stated requirement?

A. Recoverability B. Security C. Availability D. Manageability

Here’s why:

Question 45

A VMware Cloud Foundation multi-AZ (Availability Zone) design requires that:

All management components remain centralized. The availability SLA must be no less than 99.99%.

Which two design decisions would help meet these requirements? (Choose two.)

A. Implement a stretched L2 VLAN for the infrastructure management components between the AZs. B. Select two distant AZs and configure separate management workload domains. C. Implement VMware Live Recovery between the selected AZs. D. Implement separate VLANs for the infrastructure management components within each AZ. E. Select two close proximity AZs and configure a stretched management workload domain.

A, E

Here’s why:

Conclusion: C: VMware Live Recovery enables centralized management with DR failover, supporting 99.99% availability. E: A stretched management domain in close AZs ensures centralized, highly available management with near-zero downtime. These decisions align with VCF 5.2 multi-AZ best practices. Reference: VMware Cloud Foundation 5.2 Architectural Guide (docs.vmware.com): Multi-AZ Design and Stretched Clusters. VMware Cloud Foundation 5.2 Administration Guide (docs.vmware.com): Management Domain Resilience. VMware Live Recovery Documentation (docs.vmware.com): DR for VCF Environments.

Question 46

A customer has stated the following requirements for Aria Automation within their VCF implementation:

Users must have access to specific resources based on their company organization.

Developers must only be able to provision to the Development environment. Production workloads can be placed on DMZ or Production clusters.

What two design decisions must be implemented to satisfy these requirements? (Choose two.)

A. Separate tenants will be configured for Development and Production. B. Users’ access to resources will be controlled by tenant membership. C. Users’ access to resources will be controlled by project membership. D. Separate cloud zones will be configured for Development and Production.

C, D

Here’s why: Conclusion: C: Project membership enforces user access per organization and restricts developers to Development, meeting the first two requirements. D: Separate cloud zones isolate Development from Production/DMZ, enabling precise workload placement per the third requirement. These decisions align with Aria Automation’s design in VCF 5.2. Reference: VMware Cloud Foundation 5.2 Architectural Guide (docs.vmware.com): Aria Automation Design and Cloud Zones. VMware Aria Automation Administration Guide (docs.vmware.com): Projects and Access Control. VMware Aria Automation Installation Guide (docs.vmware.com): Tenancy Options in VCF.

Question 47

As part of the requirement gathering phase, an architect identified the following requirement for the newly deployed SDDC environment: Reduce the network latency between two application virtual machines. To meet the application owner’s goal, which design decision should be included in the design?

A. Configure a Storage DRS rule to keep the application virtual machines on the same datastore. B. Configure a DRS rule to keep the application virtual machines on the same ESXi host. C. Configure a DRS rule to separate the application virtual machines to different ESXi hosts. D. Configure a Storage DRS rule to keep the application virtual machines on different datastores.

Here’s why: Conclusion: Option B is the correct design decision. A DRS affinity rule ensures the VMs share the same host, minimizing network latency by leveraging intra-host communication, aligning with VCF 5.2 best practices for latency-sensitive workloads. Reference: VMware Cloud Foundation 5.2 Architectural Guide (docs.vmware.com): Section on DRS and Workload Placement. vSphere Resource Management Guide (docs.vmware.com): DRS Affinity Rules and Network Latency Considerations. VMware Cloud Foundation 5.2 Administration Guide (docs.vmware.com): SDDC Design for Performance

Question 48

During the requirements capture workshop, the customer expressed a plan to use Aria Operations Continuous Availability to satisfy the availability requirements for a monitoring solution. They will validate the feature by deploying a Proof of Concept (POC) into an existing low-capacity lab environment. What is the minimum Aria Operations analytics node size the architect can propose for the POC design?

A. Small B. Medium C. Extra Small D. Large

Here’s why: Conclusion: The minimum Aria Operations analytics node size for the POC is Small (A), enabling Continuous Availability in a low-capacity lab while meeting the customer’s validation goal. Reference: VMware Cloud Foundation 5.2 Architectural Guide (docs.vmware.com): Aria Operations Integration and HA Features. VMware Aria Operations Administration Guide (docs.vmware.com): Continuous Availability Configuration and Requirements.

Question 49

Which statement defines the purpose of Technical Requirements?

A. Technical requirements define which goals and objectives can be achieved. B. Technical requirements define what goals and objectives need to be achieved. C. Technical requirements define which audience needs to be involved. D. Technical requirements define how the goals and objectives can be achieved.

Here’s why: Conclusion: Option D accurately reflects the purpose of Technical Requirements in VCF 5.2, focusing on the means to achieve business objectives. Reference: VMware Cloud Foundation 5.2 Architectural Guide (docs.vmware.com): Section on Requirements Classification. VMware Cloud Foundation 5.2 Design Guide (docs.vmware.com): Business vs. Technical Requirements.

Question 50

An architect is tasked with updating the design for an existing VMware Cloud Foundation (VCF) deployment to include four vSAN ESA ready nodes. The existing deployment comprises the following:

Four homogenous vSAN ESXi ready nodes in the management domain.

Four homogenous ESXi nodes with iSCSI principal storage in workload domain A.

What should the architect recommend when including this additional capacity for application workloads?

A. Commission the four new nodes into the existing workload domain A cluster. B. Create a new vLCM image workload domain with the four new nodes. C. Create a new vLCM baseline cluster in the existing workload domain with the four new nodes. D. Create a new vLCM baseline workload domain with the four new nodes.

Here’s why: Conclusion: Option D is the best recommendation, creating a new vSAN ESA-based workload domain managed by vLCM, meeting capacity needs while adhering to VCF 5.2 storage and domain consistency rules. Reference: VMware Cloud Foundation 5.2 Architectural Guide (docs.vmware.com): Workload Domain Design and vSAN ESA. VMware Cloud Foundation 5.2 Administration Guide (docs.vmware.com): vLCM and Cluster Expansion. vSAN ESA Planning and Deployment Guide (docs.vmware.com): Storage Requirements.

Question 51

The following are a list of design decisions made relating to networking:

NSX Distributed Firewall (DFW) rule to block all traffic by default.

Implement overlay network technology to scale across data centers.

Configure Cisco Discovery Protocol (CDP) - Listen mode on all Distributed Virtual Switches (DVS).

Use of 2x 64-port Cisco Nexus 9300 for top-of-rack ESXi host switches. Which design decision would an architect document within the logical design?

A. Use of 2x 64-port Cisco Nexus 9300 for top-of-rack ESXi host switches. B. NSX Distributed Firewall (DFW) rule to block all traffic by default. C. Implement overlay network technology to scale across data centers. D. Configure Cisco Discovery Protocol (CDP) - Listen mode on all Distributed Virtual Switches (DVS).

Here’s why: Conclusion: Option C belongs in the logical design, as it defines a scalable networking architecture critical to VCF 5.2’s multi-data center capabilities. Reference: VMware Cloud Foundation 5.2 Architectural Guide (docs.vmware.com): Logical Design and Overlay Networking. VMware Cloud Foundation 5.2 Networking Guide (docs.vmware.com): NSX and DVS Configuration. VMware Cloud Foundation 5.2 Design Guide (docs.vmware.com): Logical vs. Physical Design.

Question 52

A customer is implementing a new VMware Cloud Foundation (VCF) instance and has a requirement to deploy Kubernetes-based applications. The customer has no budget for additional licensing. Which VCF feature must be implemented to satisfy the requirement?

A. Tanzu Mission Control B. VCF Edge C. Aria Automation D. IaaS control plane

Here’s why: Conclusion: The IaaS control plane (D), leveraging vSphere with Tanzu, meets the requirement for Kubernetes deployment within VCF 5.2’s existing licensing, satisfying the no-budget constraint. Reference: VMware Cloud Foundation 5.2 Architectural Guide (docs.vmware.com): IaaS Control Plane and vSphere with Tanzu. VMware Cloud Foundation 5.2 Administration Guide (docs.vmware.com): Workload Management Features. VMware Cloud Foundation 5.2 Licensing Guide (docs.vmware.com): Included Components.

Question 53

A VMware Cloud Foundation design is focused on IaaS control plane security, where the following requirements are present:

Support for Kubernetes Network Policies. Cluster-wide network policy support. Multiple Kubernetes distribution(s) support. What would be the design decision that meets the requirements for VMware Container Networking?

A. NSX VPCs B. Antrea C. Harbor D. Velero Operators

Here’s why:

Question 54

An architect is designing a new VCF solution to meet the following requirements:

The solution must be deployed across two availability zones.

The physical hosts must be installed in a single rack per availability zone.

Workloads running in the cluster must be able to run on hosts in either availability zone.

The architect has decided that to meet these requirements, the solution will be deployed using the Single Instance - Multiple Availability Zones VCF Topology. When considering the design for the network, what should the architect include in the logical design to meet these requirements?

A. A physical network fabric in a leaf-spine configuration with dual Cisco switches within each availability zone. B. A highly available gateway that supports the failure of an entire availability zone. C. A 25-GbE port on each Top of Rack (ToR) switch connected to the ESXi host uplinks. D. A single NSX Overlay Transport Zone for all clusters to carry the traffic between the ESXi hosts.

Here’s why:

Question 55

The following are a set of design decisions related to networking:

DD01: Set NSX Distributed Firewall (DFW) to block all traffic by default. DD02: Use VLANs to separate physical network functions. DD03: Connect the management interface eth0 of each NSX Edge node to VLAN 100. DD04: Deploy 2x 64-port Cisco Nexus 9300 switches for top-of-rack ESXi host connectivity. Which design decision would an architect include in the logical design?

A. DD04 B. DD01 C. DD03 D. DD02

Here’s why:

Question 56

The following requirements were identified in an architecture workshop for a VMware Cloud Foundation (VCF) design project using vSAN as the primary storage solution:

REQ001: The application must maintain a minimum of 1,000 transactions per second (TPS) during business hours, excluding disaster recovery (DR) scenarios. REQ002: Automatic DRS and HA must be utilized. REQ003: Planned maintenance must be performed outside of business hours. While monitoring the TPS of the application, which of the following is NOT a valid test case to validate these requirements?

A. Trigger a vSphere High Availability (HA) failover activity. B. Trigger a vSAN disk group cache drive failure. C. Trigger fully automatic DRS vMotion activity. D. Trigger a vCenter upgrade workflow.

Here’s why:

Question 57

A customer has a database cluster running in a VCF cluster with the following characteristics:

40/60 Read/Write ratio.

High IOPS requirement.

No contention on an all-flash OSA vSAN cluster in a VI Workload Domain. Which two vSAN configuration options should be configured for best performance? (Choose two.)

A. Flash Read Cache Reservation B. RAID 1 C. Deduplication and Compression disabled D. Deduplication and Compression enabled E. RAID 5

B,C

Here’s why:

Question 58

which three elements should be considered when calculating usable resources? (Choose three.)

A. vSAN space efficiency feature enablement B. VM swap file C. Disk capacity per VM D. Number of 10GbE NICs per VM E. CPU/Cores per VM F. Number of VMs

A,B,E

Here’s why:

Question 59

An architect is working with a service provider to design a VMware Cloud Foundation (VCF) solution that is required to host workloads for multiple tenants. The following requirements were gathered:

Each tenant requires full access to their own vCenter.

Each tenant will utilize and manage their own identity provider for access.

A total of 28 tenants are expected to be onboarded.

Each tenant will have their own independent VCF lifecycle maintenance schedule.

Which VCF architecture option will meet these requirements?

A. A single VCF instance consolidated architecture model with 28 tenant clusters B. A single VCF instance standard architecture model and 28 isolated SSO domains C. Two VCF instances consolidated architecture model with 14 tenant clusters each D. Two VCF instances with standard architecture model and 14 isolated SSO domains each

Here’s why:

Question 60

An architect is designing a VMware Cloud Foundation (VCF)-based solution for a customer with the following requirement:

The solution must not have any single points of failure.

To meet this requirement, the architect has decided to incorporate physical NIC teaming for all vSphere host servers. When documenting this design decision, which consideration should the architect make?

A. Embedded NICs should be avoided for NIC teaming. B. Only 10GbE NICs should be utilized for NIC teaming. C. Each NIC team must comprise NICs from the same physical NIC card. D. Each NIC team must comprise NICs from different physical NIC cards.

Here’s why:

Question 61

An architect is responsible for updating the design of a VMware Cloud Foundation solution for a pharmaceuticals customer to include the creation of a new cluster that will be used for a new research project. The applications that will be deployed as part of the new project will include a number of applications that are latency-sensitive. The customer has recently completed a right-sizing exercise using VMware Aria Operations that has resulted in a number of ESXi hosts becoming available for use. There is no additional budget for purchasing hardware. Each ESXi host is configured with:

2 CPU sockets (each with 10 cores)

512 GB RAM divided evenly between sockets

The architect has made the following design decisions with regard to the logical workload design:

The maximum supported number of vCPUs per virtual machine size will be 10.

The maximum supported amount of RAM (GB) per virtual machine will be 256.

What should the architect record as the justification for these decisions in the design document?

A. The maximum resource configuration will ensure efficient use of RAM by sharing memory pages between virtual machines. B. The maximum resource configuration will ensure the virtual machines will cross NUMA node boundaries. C. The maximum resource configuration will ensure the virtual machines will adhere to a single NUMA node boundary. D. The maximum resource configuration will ensure each virtual machine will exclusively consume a whole CPU socket.

Here’s why:

Question 62

A customer is designing a new VMware Cloud Foundation stretched cluster using L2 non-uniform connectivity, where due to a past incident an attacker was able to inject some false routes into their dynamic global routing table. What design decision can be taken to prevent this when configuring the Tier-0 gateway?

A. OSPF MD5 authentication B. Gateway Firewall with ECMP C. Implicit deny for any traffic D. BGP peer password

Here’s why:

Question 63

Due to limited budget and hardware, an administrator is constrained to a VMware Cloud Foundation (VCF) consolidated architecture of seven ESXi hosts in a single cluster. An application that consists of two virtual machines hosted on this infrastructure requires minimal disruption to storage I/O during business hours. Which two options would be most effective in mitigating this risk without reducing availability? (Choose two.)

A. Apply 100% CPU and memory reservations on these virtual machines B. Implement FTT=1 Mirror for this application virtual machine C. Replace the vSAN shared storage exclusively with an All-Flash Fibre Channel shared storage solution D. Perform all host maintenance operations outside of business hours E. Enable fully automatic Distributed Resource Scheduling (DRS) policies on the cluster

B,D

Here’s why:

Question 64

A VMware Cloud Foundation multi-AZ (Availability Zone) design mandates that:

All availability zones must operate independently of each other. The availability SLA must adhere to no less than 99.9%. What would be the three design decisions that would help satisfy those requirements? (Choosethree.)

A. Configure array-based replication between the selected AZ(s) for the management domain B. Make sure all configuration backups are replicated between the selected AZ(s) C. Make sure the recovery VLAN for the infrastructure management components has access to both AZ(s) D. Choose two distant AZ(s) and consider each AZ the DR for the other E. Choose two close proximity AZ(s) and configure a stretched management workload domain F. Configure a non-routable separate recovery VLAN for the infrastructure management components within each AZ

A,B,F

Here’s why:

Question 65

An architect is documenting the design for a new VMware Cloud Foundation solution. Which statement would be an example of a conceptual model for this solution?

A. A detailed description of the VMware Cloud Foundation solution configuration, including host names and IP addresses B. A detailed diagram of the interfaces of the NSX Edge components within the management domain in the data center C. A high-level diagram of the VMware Cloud Foundation solution showing the workload domains with the number of physical hosts per cluster D. A high-level overview of the solution, including risks, assumptions, and constraints

Here’s why:

Question 66

When sizing a VMware Cloud Foundation VI Workload Domain, which three factors should be considered when calculating usable compute capacity? (Choose three.)

A. NSX B. vSphere HA C. vSAN D. NIOC E. Storage DRS F. Core Dumps

B,C,D

Here’s why:

Question 67

Which statement defines the purpose of Business Requirements?

A. Business requirements define which audience needs to be involved. B. Business requirements define how the goals and objectives can be achieved. C. Business requirements define which goals and objectives can be achieved. D. Business requirements define what goals and objectives need to be achieved.

Here’s why:

Question 68

During a security-focused design workshop for a new VMware Cloud Foundation (VCF) solution, a key stakeholder described the current and potential future approach to user authentication within their organization. The following information was captured by an architect:

All users within the organization currently have Active Directory-backed user accounts.
A separate project is planned to evaluate the use of different 3rd-party identity solutions to enforce
Multi-Factor Authentication (MFA) on all user accounts.
The MFA project will only provide a recommendation on which identity solution the organization should implement.
The MFA project will need to request budget for any licenses that need to be procured for the recommended identity solution.
The new VCF environment may be deployed before the MFA project has completed and therefore must be able to integrate with both the current and any proposed future identity solutions.

Which TWO items should the architect include in their design documentation? (Choose TWO.)

A. An assumption that the new 3rd-party identity solution will be compatible with VCF B. An assumption that the MFA project will not receive budget to implement a new 3rd-party identity solution C. A requirement that VCF will integrate only with the new 3rd-party identity solution D. A risk that the new 3rd-party identity solution may not be compatible with Active Directory E. A risk that the new 3rd-party identity solution may not be compatible with VCF

C,E

Here’s why:

Question 69

As a VMware Cloud Foundation architect, you are provided with the following requirements:

All administrative access to the cloud management components must be trusted.

All cloud management components’ communications must be encrypted.

Enhancement of lifecycle management should always be considered.

Which design decision fulfills the requirements?

A. Integrate the SDDC Manager with a supported 3rd-party certificate authority (CA). B. Integrate the SDDC Manager with the vCenter Server in VMCA mode. C. Write a PowerCLI script to run on all virtual appliances and force a redirection on port 443. D. Write an Aria Orchestrator Workflow to change the ESXi hosts’ certificates in bulk.

Here’s why:

Question 70

An architect is documenting the design for a new VMware Cloud Foundation solution. During workshops with key stakeholders, the architect discovered that some of the workloads that will be hosted within the Workload Domains will need to be connected to an existing Fibre Channel storage array. How should the architect document this information within the design?

A. As an assumption B. As a constraint C. As a design decision D. As a business requirement

Here’s why:

Question 71

During a requirement capture workshop, the customer expressed a plan to use Aria Operations Continuous Availability. The customer identified two datacenters that meet the network requirements to support Continuous Availability; however, they are unsure which of the following datacenters would be suitable for the Witness Node.

Which datacenter meets the minimum network requirements for the Witness Node?

A. Datacenter A B. Datacenter B C. Datacenter C D. Datacenter D

Here’s why:

Question 72

As part of a VMware Cloud Foundation (VCF) design, an architect is responsible for planning for the migration of existing workloads using HCX to a new VCF environment. Which two prerequisites would the architect require to complete the objective? (Choose two.)

A. Extended IP spaces for all moving workloads. B. DRS enabled within the VCF instance. C. Service accounts for the applicable appliances. D. NSX Federation implemented between the VCF instances. E. Active Directory configured as an authentication source.

覺得正確答案只有C，硬要選第二個 B or E

C,E

Here’s why:

Question 73

An architect is designing a new VMware Cloud Foundation (VCF) solution. During the discovery workshops, the customer explained that the solution will initially be used to host a single business application and some internal management tooling. The customer provided the following background information:

The business application consists of two virtual machines.

The business application is sensitive to changes in its storage I/O.

The business application must be available during the company’s business hours of 9 AM - 5 PM on weekdays.

The architect has made the following design decisions in response to the customer’s requirements and the additional information provided during discovery:

The solution will use the VCF consolidated architecture model.

A single cluster will be created, consisting of six ESXi hosts.

Which design decision should the architect include in the design to mitigate the risk of impacting the business application?

A. Use resource pools to apply CPU and memory reservations on the business application virtual machines. B. Implement FTT=6 for the business application virtual machines. C. Perform ESXi host maintenance activities outside of the stated business hours. D. Replace the vSAN shared storage exclusively with an All-Flash Fibre Channel shared storage solution. E. Use Anti-Affinity Distributed Resource Scheduler (DRS) rules on the business application virtual machines.

Here’s why:

Question 74

An administrator is designing a new VMware Cloud Foundation instance that has to support management, VDI, DB, and general workloads. The DB workloads will stay the same in terms of resources over time. However, the general workloads and VDI environments are expected to grow over the next 3 years. What should the architect include in the documentation?

A. An assumption that the DB workload resource requirements will remain static. B. A constraint of including the management, DB, and VDI environments. C. A requirement consisting of the growth of the general workloads and VDI environment. D. A risk that the VCF instance may not have enough capacity for growth

Here’s why:

Question 75

An architect is sizing the workloads that will run in a new VMware Cloud Foundation (VCF) Management Domain. The customer has a requirement to use Aria Operations to provide effective monitoring of the new VCF solution. What is the minimum Aria Operations Analytics node size requirement when Aria Suite Lifecycle is in VCF-aware mode?

A. Small B. Extra Large C. Medium D. Large

Here’s why:

Question 76

An architect has been tasked with reviewing a VMware Cloud Foundation design document. Observe the following requirements:

REQ01: The solution must support the private cloud cybersecurity industry and local standards and controls.

REQ02: The solution must ensure that the cloud services are transitioned to operation teams.

REQ03: The solution must provide a self-service portal.

REQ04: The solution must provide the ability to consume storage based on policies.

REQ05: The solution should provide the ability to extend networks between different availability zones.

Observe the following design decisions:

DD01: There will be a clustered deployment of Aria Automation.

DD02: There will be an integration between Aria Automation and multiple geo-located vCenter Servers.

Based on the information provided, which two requirements satisfy the stated design decisions?(Choose two.)

A. REQ01 B. REQ02 C. REQ03 D. REQ04 E. REQ05

C,E

Here’s why:

Question 77

An architect is documenting the design for a new VMware Cloud Foundation-based solution. Following the requirements gathering workshops held with customer stakeholders, the architect has made the following assumptions:

The customer will provide sufficient licensing for the scale of the new solution.

The existing storage array that is to be used for the user workloads has sufficient capacity to meet the demands of the new solution.

The data center offers sufficient power, cooling, and rack space for the physical hosts required by the new solution.

The physical network infrastructure within the data center will not exceed the maximum latency requirements of the new solution.

Which two risks must the architect include as a part of the design document because of these assumptions? (Choose two.)

A. The physical network infrastructure may not provide sufficient bandwidth to support the user workloads. B. The customer may not have sufficient data center power, cooling, and physical rack space available. C. The customer may not have licensing that covers all of the physical cores the design requires. D. The assumptions may not be approved by a majority of the customer stakeholders before the solution is deployed.

A,C

Here’s why:

Question 78

An architect is designing a VMware Cloud Foundation (VCF)-based Private Cloud solution. During the requirements gathering workshop with customer stakeholders, the following information was captured:

The solution must be capable of deploying 50 concurrent workloads.

The solution must ensure that once submitted, each service does not take longer than 6 hours to provision.

When creating the design documentation, which design quality should be used to classify the stated requirements?

A. Availability B. Recoverability C. Performance D. Manageability

Here’s why:

Question 79

An architect is collaborating with a client to design a VMware Cloud Foundation (VCF) solution required for a highly secure infrastructure project that must remain isolated from all other virtual infrastructures. The client has already acquired six high-density vSAN-ready nodes, and there is no budget to add additional nodes throughout the expected lifespan of this project. Assuming capacity is appropriately sized, which VCF architecture model and topology should the architect suggest?

A. Single Instance - Multiple Availability Zone Standard architecture model B. Single Instance Consolidated architecture model C. Single Instance - Single Availability Zone Standard architecture model D. Multiple Instance - Single Availability Zone Standard architecture model

Here’s why:

Question 80

An architect is tasked with designing a new VMware Cloud Foundation environment and has identified the following customer-provided requirements:

REQ01: The application server must handle at least 30,000 transactions per second.

REQ02: The design must meet ISO 27001 information security standards.

REQ03: The storage network should maintain a minimum latency of 12 milliseconds before path failover.

REQ04: The staging environment should utilize a secondary third-party data center.

REQ05: Planned maintenance must be performed outside the hours of 8 AM to 8 PM GMT.

What are the two functional requirements? (Choose two.)

A. REQ01 B. REQ02 C. REQ03 D. REQ04 E. REQ05

A,D

Here’s why:

Question 81

An architect is working on higher-scale NSX Grouping and security design requirements for Management and VI Workload Domains in VMware Cloud Foundation. Which NSX Manager appliance size will be considered for use?

A. Extra Large B. Large C. Medium D. Small

Here’s why:

Question 82

An architect is planning the deployment of Aria components in a VMware Cloud Foundation environment using SDDC Manager and must prepare a logical diagram with networking connections for particular Aria products. Which are two valid Application Virtual Networks for Aria Operations deployment using SDDC Manager? (Choose two.)

A. Region-A - Overlay backed segment B. Region-A - VLAN backed segment C. X-Region - VLAN backed segment D. X-Region - Overlay backed segment

A, D

Here’s why:

Question 83

A VMware Cloud Foundation (VCF) platform has been commissioned, and lines of business are requesting approved virtual machine applications via the platform’s integrated automation portal.

The platform was built following all provided company security guidelines and has been assessed against Sarbanes-Oxley Act of 2002 (SOX) regulations. The platform has the following characteristics:

One Management Domain with a single cluster, supporting all management services with all network traffic handled by a single Distributed Virtual Switch (DVS).

A dedicated VI Workload Domain with a single cluster for all line of business applications.

A dedicated VI Workload Domain with a single cluster for Virtual Desktop Infrastructure (VDI).

Aria Operations is being used to monitor all clusters.

VI Workload Domains are using a shared NSX instance.

An application owner has asked for approval to install a new service that must be protected as per the Payment Card Industry (PCI) Data Security Standard, which is going to be verified by a third-party organization. To support the new service, which additional non-functional requirement should be added to the design?

A. The VCF platform and all PCI application virtual machines must be monitored using the Aria Operations Compliance Pack for Payment Card Industry. B. The VCF platform and all PCI application virtual machines must be assessed for SOX compliance. C. The VCF platform and all PCI application virtual machine network traffic must be routed via NSX. D. The VCF platform and all PCI application virtual machines must be assessed against Payment Card Industry Data Security Standard (PCI DSS) compliance.

Here’s why:

Question 84

An architect is working on a leaf-spine design requirement for NSX Federation in VMware Cloud Foundation. Which recommendation should the architect document?

A. Use a physical network that is configured for EIGRP routing adjacency. B. Layer 3 device that supports OSPF. C. Ensure that the latency between VMware Cloud Foundation instances that are connected in an NSX Federation is less than 1500 ms. D. Jumbo frames on the components of the physical network between the VMware Cloud Foundation instances.

Here’s why:

Question 85

An architect is designing a new VMware Cloud Foundation (VCF)-based Private Cloud solution.

During the requirements gathering workshop, a stakeholder from the network team stated that:

The solution must ensure that any physical networking component is redundant to N+N.

The solution must ensure inter-datacenter network links are diversely routed.

When writing the design documentation, how should the architect classify the stated requirement?

A. Availability B. Performance C. Recoverability D. Manageability

Here’s why:

Question 86

The following storage design decisions were made:

DD01: A storage policy that supports failure of a single fault domain being the server rack.

DD02: Each host will have two vSAN OSA disk groups, each with four 4TB Samsung SSD capacity drives.

DD03: Each host will have two vSAN OSA disk groups, each with a single 300GB Intel NVMe cache drive.

DD04: Disk drives capable of encryption at rest.

DD05: Dual 10Gb or higher storage network adapters.

Which two design decisions would an architect include in the physical design? (Choose two.)

A. DD01 B. DD02 C. DD03 D. DD04 E. DD05

B, C

Here’s why:

Question 87

An administrator is documenting the design for a new VMware Cloud Foundation (VCF) solution. During discovery workshops with the customer, the following information was shared with the architect:

All users and administrators of the solution will need to be authenticated using accounts in the corporate directory service.

The solution will need to be deployed across two geographically separate locations and run in an Active/Standby configuration where supported.

The management applications deployed as part of the solution will need to be recovered to the standby location in the event of a disaster.

All management applications will need to be deployed into a management tooling zone of the network, which is separated from the corporate network zone by multiple firewalls.

The corporate directory service is deployed in the corporate zone.

There is an internal organization policy that requires each application instance (management or end user) to detail the ports that access is required on through the firewall separately.

Firewall rule requests are processed manually one application instance at a time and typically take a minimum of 8 weeks to complete.

The customer also informed the architect that the new solution needs to be deployed and ready to start the organization’s acceptance into service process within 3 months, as it is a dependency in the deployment of a business-critical application. When considering the design for the Cloud Automation and Operations products within the VCF solution, which three design decisions should the architect include based on this information? (Choose three.)

A. The Cloud Automation and Operations products will be reconfigured to integrate with the Identity Broker solution instance at the standby site in case of a Disaster Recovery incident. B. The Identity Broker solution will be deployed at both the primary and standby site. C. The Identity Broker solution will be connected with the corporate directory service for user authentication. D. The Identity Broker solution will be deployed at the primary site and failed over to the standby site in case of a disaster. E. The Cloud Automation and Operations products will be integrated with a single instance of an Identity Broker solution at the primary site. F. The Cloud Automation and Operations products will be integrated directly with the corporate directory service.

B,C,E

Here’s why:

Question 88

A company plans to expand its existing VMware Cloud Foundation (VCF) environment for a new application. The current VCF environment includes a Management Domain and two separate VI Workload Domains with different hardware profiles. The new application has the following requirements:

The application will use significantly more memory than current workloads.

The application will have a limited number of licenses to run on hosts.

Additional VCF and hardware costs have been approved for the application.

The application will contain confidential customer information that requires isolation from other workloads. What design recommendation should the architect document?

A. Implement a new Workload Domain with hardware supporting the memory requirements of the new application. B. Deploy a new consolidated VCF instance and deploy the new application into it. C. Purchase sufficient matching hardware to meet the new application’s memory requirements and expand an existing cluster to accommodate the new application. Use host affinity rules to manage the new licensing. D. Order enough identical hardware for the Management Domain to meet the new application requirements and design a new Workload Domain for the application.

Here’s why:

Question 89

As part of a new VMware Cloud Foundation (VCF) deployment, a customer is planning to implement the vSphere IaaS control plane. What component could be installed and enabled to implement the solution?

A. Storage DRS B. Aria Automation C. Aria Operations D. NSX Edge networking

Here’s why:

Question 90

An architect was requested to recommend a solution for migrating 5000 VMs from an existing vSphere environment to a new VMware Cloud Foundation infrastructure. Which feature or tool can be recommended by the architect to minimize downtime and automate the process?

A. VMware HCX B. vSphere vMotion C. VMware Converter D. Cross vCenter vMotion

Here’s why:

Question 57

A customer has a database cluster running in a VCF cluster with the following characteristics:

40/60 Read/Write ratio
High IOPS requirement
No contention on an all-flash OSA vSAN cluster in a VI Workload Domain

Which two vSAN configuration options should be configured for best performance? (Choose two.)

客戶在 VCF 叢集中運行一個資料庫叢集，具有以下特性：

40/60 讀/寫比率
高 IOPS 要求
在 VI Workload Domain 中的全快閃 OSA vSAN 叢集上沒有資源爭用

為了獲得最佳效能，應配置哪兩個 vSAN 設定選項？（選擇兩項）

A. RAID 1 B. Deduplication and Compression enabled C. RAID 5 D. Flash Read cache reservation E. Deduplication and Compression disabled

A、E

Here’s why: A. RAID-1 提供最低的寫入延遲及最高的吞吐量——不像 RAID-5/6 需要進行奇偶校驗運算，且讀取可由任一鏡像複本提供服務。

E. 雖然去重／壓縮能節省容量，但在快取刷寫（destaging）過程中會引入 CPU 與 I/O 負擔，進而降低實際寫入快取的刷寫吞吐量。對於以寫入為主、高 IOPS 的資料庫，停用這些功能可將效能最大化。

Question #91

An architect is sizing the workloads that will run in a new VMware Cloud Foundation (VCF) Management Domain, the customer has a requirement to use Aria Operations to provide effective monitoring of the new VCF Solution. What is the minimum Aria Operations Analytics node size requirement when Aria Suite Lifecycle is in VCF aware mode?

一位架構師正在為即將在新的 VMware Cloud Foundation（VCF）管理域中執行的工作負載進行規模評估，客戶要求使用 Aria Operations 來有效監控新的 VCF 解決方案。當 Aria Suite Lifecycle 處於 VCF 感知模式時，Aria Operations Analytics 節點的最低規模需求是多少？

A. Extra Large B. Small C. Medium D. Large

Here’s why: 對於 VMware Aria Operations 的生產部署，請依據規模需求與可用資源，將所有節點部署為 medium (中型)、large (⼤型) 或 extra-large (超⼤型)。

參考連結 :

Scalability Considerations - VMware Aria Operations 8.17.1

Question #67

Which statement defines the purpose of Business Requirements?

哪一項敘述定義了業務需求的目的？

A. Business requirements define how the goals and objectives can be achieved. B. Business requirements define which goals and objectives can be achieved. C. Business requirements define what goals and objectives need to be achieved. D. Business requirements define which audience needs to be involved.

Here’s why: 業務需求闡述組織希望透過此解決方案達成的目標，例如改善應用程式效能、確保災難復原或支援特定數量的工作負載。在 VMware Cloud Foundation 5.2 的情境中，範例可能包括「此解決方案必須能支援 500 部虛擬機器」或「環境必須提供 99.99% 的運行時間」。這些敘述定義了目標與宗旨，但並未說明要如何實現（例如透過 vSphere HA 或 vSAN）或由誰來執行。這符合 VMware 的設計方法論：由業務需求驅動後續的功能性與非功能性需求。在 VMware Cloud Foundation 5.2 中，架構設計流程就是從蒐集業務需求開始，以確保解決方案能符合組織需求。VMware Cloud Foundation Planning and Preparation Guide 強調，業務需求建立了「做什麼」（例如降低成本或整合工作負載等期望成果），進而影響技術架構的規劃，例如 VI 工作負載域的規模或管理元件的部署。

Question #4

The application will use significantly more memory than current workloads today.
The application will have a limited number of licenses to run on hosts.
Additional VCF and hardware costs has been approved for the application.
The application will contain confidential customer information that requires isolation from other workloads.

What design recommendations should the administrator document?

一家公司將擴展其現有的 VCF 環境，以部署一個新應用程式。現有的 VCF 環境目前有一個管理域，以及兩個具有不同硬體配置的獨立 VI 工作負載域。該新應用程式具有以下需求：

應用程式將使用顯著多於目前工作負載的記憶體。
應用程式在主機上的授權數量有限。
已批准針對該應用程式支付額外的 VCF 及硬體成本。
應用程式將包含需要與其他工作負載隔離的機敏客戶資訊。

管理員應記錄哪些設計建議？

A. Purchase enough matching hardware to accommodate the new applications memory requirements and expand an existing cluster to accommodate the new application. Use host affinity rules to manage the new licensing. B. Enough identical hardware for the management domain should be ordered to accommodate the new application requirements and a new workload domain should be designed for the application. C. Deploy a new consolidated VCF instance and deploy the new application into it. D. A new Workload domain with hardware supporting the memory requirements of the new application should be implemented.

Here’s why: 需求包括記憶體容量、授權管控、成本核准與隔離。

選項 D「使用支援記憶體需求之硬體建立新的工作負載域」可同時滿足所有要求：在 VCF 5.2 中建立新的 VI 工作負載域，可透過獨立的 NSX 實例來隔離工作負載；使用已核准的資金購置高記憶體硬體；並且可在此領域內透過 DRS 親和性規則來管理授權。

選項 C（建立新的 VCF 實例）過於冗餘，會重複產生管理負擔；選項 B（直接擴充管理域硬體）則誤用了管理域的目的；選項 A（擴充現有叢集）則可能造成隔離失效。選擇 B 最能有效運用 VCF 工作負載域架構。參考資料：VMware Cloud Foundation 5.2 Architecture and Deployment Guide（Workload Domain Design）；VMware vSphere 7.0 Documentation（DRS Affinity Rules）。

VMware Cloud Foundation 5.2 Administrator

Sun, 08 Jun 2025 08:18:22 +0800

最後更新於 2025 年 6 月 7 日。

Question #1

What is a symptom of an issue with vSphere HA?

A. Snapshots cannot be created or consolidated. B. VMs are not restarted after a host failure. C. VMs are not being migrated using vMotion. D. Hosts frequently disconnect from vCenter Server.

Here’s why:

Question #2

Which prerequisite task must be completed before deploying VMware Cloud Foundation (VCF) using VMware Cloud Builder?

A. Ensure that all ESXi hosts are connected to a single vCenter Server. B. Verify network configurations and DNS settings for all components. C. Verify configurations on Virtual Distributed Switch on the ESXi hosts that will be used for the deployment. D. Ensure HCX is configured for workload migration.

Here’s why:

Question #3

The administrator has been tasked with applying asynchronous software updates to the VMware Cloud Foundation (VCF) instance using the VMware Async Patch CLI Tool. Which four steps would an administrator take to complete the task? (Choose four.)

A. Download the bundles from SDDC Manager. B. Download the bundles. C. Upload the confirmation to SDDC Manager. D. Apply the patches directly through the Async Patch CLI Tool. E. Enable the bundles. F. Apply the patches through the SDDC Manager UI. G. Upload the bundles to SDDC Manager.

B, D, E, G

Here’s why:

Question #4

While deploying the first VI workload domain in a VMware Cloud Foundation (VCF) solution, which two steps will always be performed by SDDC Manager? (Choose two.)

A. Deploy an NSX Edge cluster and configure BGP routing on the T0 Gateway. B. Deploy a new NSX Manager instance for the VI workload domain. C. Configure vSAN as a principal storage for the VI workload domain. D. Deploy and configure a new vCenter Server instance for the VI workload domain. E. Connect a new vCenter Server instance to the Application Virtual Network.

B, D

Here’s why:

Question #5

What is the primary benefit of using HCX Replication Assisted vMotion (RAV) for VM migration?

A. It requires no configuration changes to the VM(s) within the destination environment. B. It performs a test migration to ensure compatibility. C. It ensures zero downtime to the VM(s). D. It provides the fastest migration method by powering off the VM(s) before initiating the migration.

Here’s why:

Question #6

An administrator has been tasked with deploying a new VMware Cloud Foundation (VCF) environment and is preparing the physical hosts that will be used for the management domain. The administrator has completed the following tasks for all of the physical hosts: The required version of ESXi has been installed. Networking has been configured for the ESXi management interface. DNS entries have been created for forward and reverse name resolution. NTP has been configured and the time synchronized with a centralized time source. VLAN 10 has been presented to the ESXi hosts on the physical network infrastructure for the VCF ESXi management network. A combination of which two additional tasks must the administrator complete before starting the bring-up of the management domain? (Choose two.)

A. Upload the Cloud Builder appliance OVA file to the new shared NFS datastore. B. Create a new vSphere Standard Switch for the VCF ESXi management network on each ESXi host. C. Configure a new shared NFS datastore on each ESXi host. D. Replace the SSL certificate with either a CA signed or self-signed certificate on each ESXi host. E. Configure VLAN 10 on the VM network Port Group of the vSphere Standard Switch on each ESXi host.

D, E

Here’s why:

Question #7

How do virtual machine (VM) storage policies in a VMware vSphere environment contribute to optimizing resource utilization, ensuring data availability, and meeting application-specific requirements?

A. By guaranteeing storage IOPs for a particular virtual machine B. By regulating and optimizing storage traffic between virtual machines C. By streamlining the process of automating virtual machine backups D. By defining and enforcing storage requirements, such as performance, availability, and redundancy

Here’s why:

Question #8

An administrator configures a Local Content Library on vCenter Server A, enables publishing, and creates Subscribed Content Libraries on vCenter Server B and vCenter Server C using the publishing URL. However, the contents of the subscribed libraries on vCenter B and C are not synchronizing with the source library on vCenter A. Which step should the administrator take to troubleshoot the issue?

A. Recreate the Subscribed Content Libraries on vCenter B and C with a new publishing URL from vCenter Server A to reset the synchronization configuration. B. Verify that network connectivity exists between vCenter Server A and the other two vCenters, and ensure that the required ports for Content Library synchronization are open. C. Enable bidirectional synchronization on the Content Library settings to ensure updates are propagated back and forth between vCenter Server A and the subscribed libraries. D. Manually export the library content from vCenter Server A and import it into the subscribed libraries on vCenter B and C to initialize synchronization.

Here’s why:

Question #9

An administrator is using VMware Aria Automation to automate the provisioning of virtual machines in their VMware Cloud Foundation (VCF) environment. They notice that the deployment requests are failing frequently. What step should the administrator take in VMware Aria Operations for Logs to diagnose the cause of the failures?

A. Check the Audit Logs to see if there are any unauthorized access attempts. B. Review the Alert Definitions to ensure alerts for request failures are configured and enabled. C. Access the Dashboard page to get an overview of system health and performance metrics. D. Use the Interactive Analytics feature to search for error messages related to the requests.

Here’s why:

Question #10

Which two steps should an administrator perform to configure the backup of the VMware Cloud Foundation (VCF) SDDC Manager and NSX components to allow them to be recovered in the event of a full site failure? (Choose two.)

A. Backup the NSX Manager configuration by exporting it to an archive file. B. Configure the backup settings in SDDC Manager. C. Configure image-based backup for the NSX Manager cluster. D. Prepare an external SFTP server to store file-based backups. E. Define a scheduled task in vCenter to create a snapshot of the NSX Edge cluster.

B, D

Here’s why:

Question #11

A Cloud Administrator has been tasked with verifying all of the prerequisites are in place prior to deploying VMware Aria Suite Lifecycle. The following information has been provided for the Aria Suite Lifecycle implementation: hostname: lcm domain: vcf.company.com

IP address: 10.0.0.150 /24 -

IP gateway: 10.0.0.1 - Which three items must the Cloud Administrator ensure are available prior to deploying VMware Aria Suite Lifecycle? (Choose three.)

A. Connectivity between Application Virtual Networks, Management Network, and network services (DNS and NTP) B. A DNS A Record for lcm.vcf.company.com that points to 10.0.0.150 C. The IP address for the Standalone Tier 1 Load Balancer for VMware Aria components is prepared D. A DNS PTR Record for 10.0.0.1 that points to lcm.vcf.company.com E. A CA-signed SSL certificate for Aria Suite Lifecycle added to SDDC Manager

A, B, E

Here’s why:

Question #12

An administrator is configuring a newly provisioned workload domain within a VMware Cloud Foundation (VCF) environment. The workload domain will contain virtual machines hosting applications which are considered business critical, and the administrator wants to ensure that the virtual machines are given priority over other network traffic types. Which three steps should the administrator perform to achieve this goal? (Choose three.)

A. Configure a bandwidth resource allocation for the virtual machine traffic type. B. Enable Network I/O control (NIOC) on the workload domain vSphere Distributed Switch (vDS). C. Create a new vSphere Distributed Switch (vDS) specifically for virtual machine traffic. D. Configure dedicated physical network adapters for the ESXi management traffic. E. Disable bandwidth resource allocations for all other traffic types to ensure virtual machine traffic gets full bandwidth. F. Configure a new network resource pool to assign shares, limits and reservations for the virtual machine traffic.

A, B, F

Here’s why:

Question #13

An administrator needs to ensure that all ESXi hosts in a cluster have a consistent configuration. Which three steps are necessary to achieve this? (Choose three.)

A. Attach the host profile to all hosts in the cluster. B. Extract the host profile from the configured reference host. C. Right-click the host profile from the reference host and select duplicate host profile. D. Configure the desired settings on an ESXi host. E. Detach the host profile from all hosts in the cluster. F. Manually configure each host to match the reference host settings.

A, B, D

Here’s why:

Question #14

An administrator is tasked to configure a Content Library to share its contents across multiple vCenter Server instances. What are the necessary steps and considerations to ensure the library is accessible for the consumption by the other vCenter instances?

A. Create a Local Content Library on each vCenter Server instance and manually synchronize the content. B. Create a Subscribed Content Library on each vCenter Server instance without enabling synchronization. C. create a Local Content Library on a primary vCenter Server instance and enable the “Publish” option. D. create a Subscribed Content Library on the primary vCenter instance and enable the “Publish” option.

Here’s why:

Question #15

Which feature of VMware Data Services Manager enhances security and compliance?

A. Automated workload migration B. Network traffic encryption C. Storage replication D. Database level authentication

Here’s why:

Question #16

An administrator is tasked with deploying a new, highly-available application on a VMware Cloud Foundation (VCF)-based private cloud. The new application will include both virtual machines and containers. Which two components of VCF could the administrator use to complete the task? (Choose two.)

A. HCX B. Aria Suite Lifecycle C. NSX D. Aria operations E. vSphere Supervisor

C, E

Here’s why:

Question #17

What are three prerequisites for deploying an NSX Edge cluster in a VMware Cloud Foundation (VCF) solution? (Choose three.)

A. Set up vSAN storage policies for the Edge VMs. B. Verify that the NSX host overlay VLAN and NSX Edge overlay VLAN are routed to each other. C. Create DNS entries for the NSX Edge nodes. D. Use separate VLAN IDs and subnets for the NSX host overlay and NSX Edge overlay networks. E. Configure BGP as a dynamic routing protocol on the physical switches. F. Install Aria Operations and enable Edge Cluster monitoring.

B, C, D

Here’s why:

Question #18

What is the primary use case for Aria Operations within VMware Cloud Foundation (VCF)?

A. To provide comprehensive monitoring, analytics, and performance management of the cloud infrastructure B. To automate workload deployments across multiple cloud environments C. To enable network virtualization and security D. To manage and analyze log data from various sources

Here’s why:

Question #19

An administrator is tasked with enabling Workload Management (vSphere Supervisor) on a VMware Cloud Foundation (VCF) workload domain using the SDDC Manager UI. Which three are prerequisites for enabling Workload Management? (Choose three.)

A. Verify that the vSphere Distributed Switch is configured with sufficient port groups. B. Verify at least one Content Library is available. C. Verify all hosts within the selected Sphere clusters have the proper vSphere with Tanzu licensing to support Workload Management. D. Verify at least one NSX Edge cluster is deployed and available. E. Verify that the cluster is running vSAN.

B, C, D

Here’s why:

Question #20

What three steps are required to commission a new host into the SDDC Manager inventory? (Choose three.)

A. Self-signed certificate regenerated based on FQDN of host. B. Four NIC ports with a minimum 10 Gbps speed. C. Delete all disk partitions on HDD/SSD. D. Update the DNS records and add the host directly to the SDDC Manager appliance using the root credentials. E. Manually import the host into the vCenter Server inventory and assign it to a cluster. F. A supported version of ESXi installed on the host.

A, C, F

Here’s why:

Question #21

An administrator recently incorrectly deployed a new isolated VI workload domain within an existing VMware Cloud Foundation (VCF) instance. How can the administrator reconfigure the VI workload domain to share the SSO domain from the management workload domain?

A. From the management domain vCenter Server instance, use the rsautil command to add the VI workload domain to the management domain. B. From the VI workload domain vCenter Server instance, add the management domain SSO and remove the isolated SSO. C. From the SDDC Manager console, migrate the VI workload domain to the management domain. D. A new VI workload domain must be created through SDDC Manager.

Here’s why:

Question #22

Following an internal security audit of the new VMware Cloud Foundation (VCF) instance, the following audit finding was documented for priority remediation: All users from the custom administrators group could access the Direct Console User Interface (DCUI) on all ESXi hosts within the workload domain. RISK=High, IMPACT=High The company IT security policy around accessing ESXi servers states the following: Users within the custom administrators group must access ESXi host configurations from within vCenter Server or the vSphere Web Client only. Only users within the restricted administrators group must be allowed direct access to ESXi hosts. Which two actions should the administrator perform on each of the hosts within the workload domain to remediate the security finding? (Choose two.)

A. Disable SSH and the ESXi Shell. B. Add the custom administrators group to the DCUI.Access advanced system setting. C. Add the restricted administrators group to the DCUI.Access advanced system setting. D. Enable Strict Lockdown Mode. E. Enable Normal Lockdown Mode.

C, E

Here’s why:

Question #23

An administrator has been tasked with creating a view of key performance indicators for their VMware Cloud Foundation (VCF) private cloud and an automated monthly report that shows the capacity consumed by each development team who use the environment. Which VCF component should the administrator configure to complete their objectives?

A. VMware Aria Operations for Logs B. VMware Aria Operations C. VMware Aria Operations for Networks D. VMware Aria Operations for Applications

Here’s why:

Question #24

An administrator has deployed a new VI Workload Domain and must deploy a Kubernetes cluster on the vSphere Supervisor. Which three steps should be followed to successfully deploy the Kubernetes cluster? (Choose three.)

A. Configure a vSphere Namespace and assign resource quotas. B. Deploy a vSphere Pod Service. C. Create a new VM template for the Kubernetes nodes. D. Deploy an NSX Edge Cluster. E. Enable Workload Management on the vSphere Cluster within SDDC Manager.

A, D, E

Here’s why:

Question #25

An organization is using VMware Aria Automation for automating virtual machine deployments. Recently, some deployment requests have been failing. An administrator needs to use Aria Automation to troubleshoot these failures. Which two steps should be taken? (Choose two.)

A. Disable integrations and re-run deployment. B. Cheek the request logs for error messages. C. Disable all automation policies temporarily. D. Review the provisioning diagrams for patterns of failure. E. Re-run the failed deployments without any changes.

B, D

Here’s why:

Question #26

An administrator is responsible for maintaining a VMware Cloud Foundation (VCF) instance. The administrator has been tasked with scaling the CPU and RAM resources of a virtual machine. Which three configuration limitations must the administrator consider when completing the task? (Choose three.)

A. To add Memory to a powered on virtual machine, Memory Hot Add must be enabled. B. To remove Memory to a powered on virtual machine, Memory Hot Add must be enabled. C. To add vCPUs to a powered on virtual machine CPU Hot Add must be enabled. D. To enable Memory Hot Add feature, the virtual machine compatibility must be ESXi 7.0 Update 2 or later. E. To add vCPUs to a powered down virtual machine, CPU Hot Add must be enabled. F. To enable the CPU Hot Add or Memory Hot Add feature, the virtual machine must be powered off.

A, C, F

Here’s why:

Question #27

What is the recommended method for IP address allocation for the management interface of VMware ESXi hosts in a VMware Cloud Foundation (VCF) environment?

A. Dynamic IP addresses assigned via DHCP without any reservations B. Dynamic IP addresses assigned via DHCP with a reservation based on MAC address C. Static IP addresses assigned manually D. Static IP address assigned using VMware NSX IP Pools

Here’s why:

Question #28

An administrator is enabling VMware’s Virtual Machine Encryption for an existing VMware Cloud Foundation (VCF) Workload Domain in order to meet their organization’s compliance regulations. Which three steps should the administrator take to complete this task? (Choose three.)

A. Configure a Key Management Server (KMS) and add it to the vCenter. B. Enable SSH on the ESXi hosts to manage encryption keys. C. Create an encryption policy in vCenter. D. Apply the encryption policy to the ESXi hosts. E. Apply the encryption policy to the existing VMs. F. Enable the encryption feature on the ESXi hosts.

A, C, E

Here’s why:

Question #29

An administrator needs to ensure that specific virtual machines within a VMware Cloud Foundation (VCF) environment use storage that can tolerate at least two host failures within a vSAN cluster. Due to financial limitations, the configuration used must prioritize providing the highest level of usable disk space for the datastore. Which three steps should be performed to meet the requirements? (Choose three.)

A. Reconfigure the existing Default Storage Policy in the vSphere Client. B. Apply the storage policy to the target VMs. C. Set Failures to Tolerate to “2 failures - RAID-6 (Erasure Coding)” in the policy configuration. D. Create a new VM Storage Policy in the vSphere Client. E. Set Failures to Tolerate to “2 failures - RAID-1 (Mirroring)” in the policy configuration. F. Configure the policy to use thin provisioning.

B, C, D

Here’s why:

Question #30

An administrator is managing a VMware Cloud Foundation (VCF) environment. To support virtual machine backup using a third-party tool, the administrator has been tasked with assigning a service account (srvbackup01) access to the Production Virtual Machine (VM) folder and all child objects. The company security policy dictates that only the minimum required permissions must be assigned to service accounts to ensure they can function as designed. Which three actions should the administrator take in the vSphere Client to complete the objective? (Choose three.)

A. Select the necessary privileges for the backup_service role. B. Assign the backup_service role to the srvbackup01 user on the Production VM folder only. C. Assign the Administrator role to the srvbackup01 user on the Production VM folder and propagate to all child objects. D. Create a new Role called backup_service. E. Assign the backup_service role to the srvbackup01 user on the Production VM folder and propagate to all child objects. F. Clone the Administrator Role to create the backup_service role.

A, D, E

Here’s why:

Question #31

A virtual private cloud administrator is regularly tasked to provision and manage new application workloads with dedicated subnets. Which two capabilities of VMware Cloud Foundation (VCF) can be used to alleviate the workload of the administrator with minimal interaction with the networking team? (Choose two.)

A. Configure vSAN storage policies for each application. B. Deploy Aria Automation to enable self-service provisioning. C. Use vSphere Resource Pools to allocate networking resources to different applications. D. Use of NSX overlay networks.

B, D

Here’s why:

Question #32

While troubleshooting a vSphere HA c luster issue, an administrator observes that a host is in the Network Isolated state. The host is unable to ping the configured isolation address and cannot access any other vSphere HA agents. Which step would resolve this issue while minimizing downtime for running virtual machines (VMs)?

A. Power off all VMs on the isolated host, remove the host from the cluster, and add it back after the network is fixed. B. Reconfigure the host’s HA agent to bypass the isolation check and allow VM operations to continue. C. Reconfigure the isolation addresses to ensure the host can ping the default gateway or other configured IPs. D. Restart the vSphere HA services on the isolated host to trigger the election of a new primary agent.

Here’s why:

Question #33

Which step must an administrator take to configure Application Virtual Networks (AVNs) from SDDC Manager while preparing to deploy VMware Aria Suite Lifecycle?

A. Deploy NSX Edge Cluster. B. Ensure that an NSX load balancer is available. C. Enable AVNs in the vCenter Server. D. Assign AVNs to specific ESXi hosts.

Here’s why:

Question #34

An administrator needs to scale an NSX Edge cluster by adding additional Edge nodes. Which three steps must be taken before scaling an Edge cluster from SDDC Manager? (Choose three.)

A. Assign a unique management IP for each Edge node. B. Assign two Tunnel Endpoint (TEP) addresses for each Edge node. C. Add an additional NSX VDS uplink for each Edge node being added. D. Verify DNS records have been added for each new Edge node. E. Place the Edge cluster in maintenance mode prior to expanding.

A, B, D

Here’s why:

Question #35

An organization is planning to manage a diverse set of databases across multiple VMware Cloud Foundation (VCF) environments using Data Services Manager. Which three capabilities of Data Services Manager would help the organization managing these databases efficiently? (Choose three.)

A. Centralized monitoring and alerting for all managed databases B. Automated database migration between on-premises and cloud environments C. Policy-based backup and recovery for databases D. Integration with vSAN for optimized storage management E. Automated database provisioning and deployment

A, C, E

Here’s why:

Question #36

An administrator is troubleshooting a high CPU usage issue of the SDDC Manager VM. Access to SDDC Manager UI is exhibiting degraded performance. What two steps should the administrator take to diagnose the issue? (Choose two.)

A. Reboot the SDDC Manager VM to clear any temporary issues. B. Login to SDDC Manager using console with root credentials, run the top command to identify any services that may be consuming high %CPU. C. Upgrade the vSphere version to the latest release. D. Review the CPU usage of the ESXi management domain hosts to identify any that are overcommitted. E. Reconfigure the DRS settings to be more aggressive in load balancing.

B, D

Here’s why:

Question #37

An administrator needs to perform lifecycle management for their VMware Cloud Foundation (VCF) components, including upgrading SDDC Manager. Which three steps should be followed to upgrade VCF? (Choose three.)

A. Perform a pre-check to ensure that all components are compatible with the new version. B. Backup all VMware Cloud Foundation components before starting the upgrade. C. Use the SDDC Manager UI to initiate the upgrade process. D. Manually download the update bundles with a secure browser from the VMware website. E. Disable DRS and HA on all clusters before initiating the upgrade.

A, B, C

Here’s why:

Question #38

An administrator is deploying a new VMware Cloud Foundation (VCF) environment. After uploading the completed Deployment Parameter Workbook, the validation task fails and an error message is displayed within the VMware Cloud Builder GUI. Which log file can the administrator use to identify the cause of the validation error?

A. vcf-bringup-debug.log B. sos.log C. domainmanager.log D. jsongenerator-<timestamp>.log

A, B, C

Here’s why:

Question #39

Which tool is most appropriate for diagnosing and resolving a disk failure when using VMware vSAN with Deduplication enabled?

A. vSphere Client B. Aria Operations C. vSphere CLI (esxcli) D. vSAN Skyline Health

Here’s why:

Question #40

An administrator is tasked with ensuring the security of network traffic during the migration of virtual machines (VMs) to prevent any potential interception or tampering of data. What specific feature or configuration should the administrator enable to safeguard the integrity and confidentiality of the network traffic involved in VM migration?

A. Encrypted vSphere vMotion B. vSphere Virtual Machine Encryption C. vSphere High Availability (HA) D. vSphere Distributed Resource Scheduler (DRS)

Here’s why:

Question #41

An administrator can set resource limits and container defaults on a vSphere Namespace. Which three resource limits can be set? (Choose three.)

A. Storage B. Memory C. CPU D. Number of containers E. Network

A, B, C

Here’s why:

Question #42

What are two benefits of enabling workload Management on a vSphere Cluster within a VI Workload Domain using SDDC Manager? (Choose two.)

A. It allows the deployment of traditional VMs with enhanced performance. B. It integrates the vSphere supervisor with NSX Networking. C. It enhances the security of the vSphere cluster. D. It integrates vSphere with third-party cloud providers. E. It enables the use of Kubernetes for container orchestration on the vSphere cluster.

B, E

Here’s why:

Question #43

An administrator finds that one of the VMs in the vSphere environment cannot reach its default gateway. The VM is configured with a static IP address, and other VMs on the same port group and VLAN are not experiencing any issues. What should the administrator check to resolve the issue?

A. Check the subnet mask configured on the VM. B. Ensure that the VM’s network adapter is set to DHCP. C. Check if the VM has the correct DNS server settings. D. Confirm that the DVS port group is configured correctly.

Here’s why:

Question #44

When setting up a new port group on a Virtual Distributed Switch (VDS), what role does configuring a VLAN ID play in network management and traffic isolation?

A. To improve the performance of virtual machines by distributing network traffic across multiple uplinks B. To logically segment network traffic by assigning the port group to a specific VLAN, ensuring that traffic is isolated C. To configure storage replication settings for virtual machines connected to the port group D. To enable the automatic assignment of IP addresses to virtual machines in the port group

Here’s why:

Question #45

An administrator has been tasked with adding a new VI workload Domain to an instance of VMware Cloud Foundation (VCF). The administrator only has two prepared ESXi host servers showing in SDDC Manager. The new VI Workload Domain must use vSphere Lifecycle Images. Which three supported principal storage options are available to the administrator when configuring the new VI Workload Domain? (Choose three.)

A. VSAN B. vVols C. iSCSI D. NFS v3 E. VMFS on FC F. VMFS on local disk

B, D, E

Here’s why:

Question #46

What are the three prerequisites an administrator must meet to deploy VMware Aria Suite Lifecycle using SDDC Manager? (Choose three.)

A. Ensure the Management Domain was deployed using vSphere Lifecycle Manager Images. B. Ensure the VMware Software Install Bundle for VMware Aria Suite Lifecycle is in the local bundle repository. C. Ensure that a custom SSL certificate specifying the FQDN of the appliance is available. D. Ensure that the NTP and DNS settings specified in SDDC Manager are valid for all management components. E. Ensure that the Application Virtual Networks have been configured.

B, D, E

Here’s why:

Question #47

A company plans to enhance its DevOps practices by implementing Kubernetes as part of a VMware Cloud Foundation (VCF) environment. Which three steps must an administrator perform to achieve this integration? (Choose three.)

A. Configure Aria Automation to automate the deployment of Kubernetes clusters. B. Enable IaaS control plane on existing vSphere clusters. C. Deploy HCX to migrate existing VMs to Kubernetes. D. Configure storage policies for Kubernetes cluster namespace consumption. E. Deploy NSX to manage the networking for Kubernetes clusters.

B, D, E

Here’s why:

Question #48

An administrator has been tasked with updating the version of VMware Tools on virtual machines that have been migrated into a VMware Cloud Foundation (VCF) instance so that they are compliant with the company’s security policies. The following information is shared with the administrator about this task: The VMs have been migrated from environments running earlier releases of vSphere. The VMs are spread across multiple clusters within a single workload domain in the VCF instance. The upgrade must not initiate a reboot of the VMs to avoid downtime to the business. Which three steps should the administrator perform in order to complete this task? (Choose three.)

A. Select the VI workload domain so that all VMs across all clusters are automatically selected. B. Use vSphere Lifecycle Manager in the VI workload domain vCenter to initiate the upgrade. C. Select the cluster hosting the VMs to upgrade. D. Select the Upgrade to Match Host option. E. Use SDDC Manager in the management domain to initiate the upgrade. F. Select the Set Auto Update option and set its value to On.

B, D, F

Here’s why:

Question #49

An administrator is tasked with deploying a new VMware Cloud Foundation (VCF) instance. The management domain is designed to be created with three virtual distributed switches (VDS) configured as follows: VDS1 - used for management and vMotion traffic

VDS2 - used for vSAN traffic - VDS3 - used for NSX Overlay traffic What action would the administrator take to deploy the designed configuration?

A. A JSON file must be prepared with the defined configuration and used to run the bring-up process. B. vSphere Distributed Switch Profile 2 must be selected in the VCF Deployment parameter workbook. C. The design must be changed to ensure vSAN and NSX overlay traffic are on the same VDS. D. The ESXi hosts have to be configured with twelve physical NICs to ensure redundancy.

Here’s why:

Question #50

Which two operations can be completed in the SDDC Manager UI on an NSX Edge cluster after it has been deployed into a workload domain? (Choose two.)

A. Redeploy B. Expand C. Sync D. Delete E. Shrink

B, E

Here’s why:

Question #51

An administrator is unable to log in to the SDDC Manager UI on their local workstation. After entering the correct URL, the login page does not load and the administrator is unable to enter their credentials in order to view the SDDC Manager inventory. They confirm that the SDDC Manager appliance responds to a ping command from their local workstation against both its FQDN and IP address. After checking the VM console of the SDDC Manager appliance in the management vCenter UI, they determine that the operating system is online and they can log in to it with the relevant administrator level accounts. What two actions could the administrator take whilst logged in to the SDDC Manager VM via the VM console to help identify the root cause of the UI not loading? (Choose two.)

A. Verify that port 80 is open and listening for connections using the command netstat. B. Perform a restart of the network adapter using the command systemctl restart. C. Verify the health of the SDDC Manager services using the command sos –get-vcf-services-summary. D. Perform a restart of the services using the command systemctl restart. E. Verify that there is free disk space remaining using the command df -h.

C, E

Here’s why:

Question #52

A single VM is unable to communicate with other VMs on the same network. An administrator needs to identify and resolve the network connectivity issue. What three steps should the administrator follow? (Choose three.)

A. Verify that the networking within the VMs guest OS is correct. B. Restart the VM to reset its network connections. C. Verify that the virtual network adapter is present and connected. D. Ensure that the Port Group name(s) associated with the VMs network adapter(s) exists in the vSwitch or VDS and is spelled correctly. E. Use the ESXi command line to check the status of the VM Kernel adapter.

A, C, D

Here’s why:

Question #53

An organization is integrating VMware vCenter with Active Directory (AD) to streamline user authentication. As part of this process, the administrator needs to add AD as an identity source in vCenter. Which three steps must be performed to successfully add AD as an identity source? (Choose three.)

A. Enter the Domain Name and the credentials of an AD user with a minimum of read-only access to Base DN for users and groups. B. Select ‘Identity Sources’ and choose ‘Active Directory Domain’. C. Configure DNS settings on all ESXi hosts to point to the AD DNS servers. D. Navigate to the vCenter Single Sign-On configuration in the vSphere Client. E. Create an Active Directory object for the vCenter Server Appliance. F. Reboot the vCenter to apply the identity source settings.

A, B, D

Here’s why:

Question #54

Which three tools could identify and assist with troubleshooting a misconfigured vSAN network? (Choose three.)

A. vCheck B. Supportability and Serviceability (SoS) C. Direct Console User Interface (DCUI) D. esxcli E. vmkping F. Ruby vSphere Console (RVC) commands

D, E, F

Here’s why:

Question #55

To help troubleshoot an issue with the vSphere Supervisor, an administrator must log into the Supervisor Control Plane VMs. Which three actions should the administrator take to meet the objective? (Choose three.)

A. Log into the vCenter Server Appliance using SSH. B. Log into the Supervisor Control Plane VM using the admin user and <password>. C. Log into the SDDC Manager appliance using SSH. D. Log into the Supervisor Control Plane VM using the root user and <password>. E. Update the <password> by running /usr/lib/vmware/auth/bin/chpw admin. F. Obtain the <password> by running /usr/lib/vmware-wcp/decryptK8Pwd.py.

A, B, F

Here’s why:

Question #56

What option should be selected when placing a vSAN host into maintenance mode to ensure that all data remains accessible throughout the maintenance process while also minimizing the duration of the operation?

A. Ensure Data Accessibility B. Ensure Accessibility C. Full Data Migration D. No Data Migration

Here’s why:

Question #57

Following an update to the Information Security policy, an administrator has been reviewing the status SSL certificates within the VMware Cloud Foundation (VCF) solution. The new Information Security Policy states: All SSL certificates must be generated and signed from the shared Microsoft Certificate Authority (CA). The administrator has discovered the following: All Aria Suite Components already use CA-signed Subject Alternate Name (SAN) SSL certificates. All other VCF-based SSL certificates are either self-signed or generated using the VMware Certificate Authority (VMCA). Which three steps must the administrator take to ensure the VCF solution remains compliant and managed by SDDC Manager? (Choose three.)

A. In VMware vCenter, replace the ESXi SSL certificates. B. Integrate the OpenSSL CA into SDDC Manager. C. Integrate the Microsoft CA into SDDC Manager. D. In SDDC Manager, replace the SSL certificates for vCenter, NSX Manager, SDDC Manager and Aria Suite Lifecycle. E. In Aria Suite Lifecycle, replace the VMware Identity Manager, Aria Automation, Aria Operations and Aria Operations for Logs SSL certificates. F. In SDDC Manager, replace the SSL certificates for vCenter, ESXi, NSX Manager, SDDC Manager and Aria Suite Lifecycle.

A, C, D

Here’s why:

Question #58

The administrator of a VMware Cloud Foundation environment notices that log data from multiple components, including vSphere, NSX, and vSAN, is not being collected in Aria Operations for Logs. What two steps should the administrator perform to resolve this issue? (Choose two.)

A. Reconfigure the log level settings on all vSphere components to ensure detailed logging. B. Check the network connectivity between the Aria Operations for Logs server and the VMware Cloud Foundation components. C. Verify that the Aria Operations for Logs license is still valid and active. D. Increase the storage capacity of the Aria Operations for Logs server to accommodate more log data. E. Ensure that the syslog configuration on each component is pointing to the correct Aria Operations for Logs server.

D, E

Here’s why:

Question #59

An organization has 3 data center sites and is implementing a solution to achieve a Recovery Point Objective (RPO) of 0 to ensure continuous availability of storage in the event of a single site failure. The sites are connected via a high bandwidth/low latency link. Which two VMware Cloud Foundation (VCF) components can be used to support this capability? (Choose two.)

A. vSphere B. Aria Operations C. NSX D. vSphere Replication E. vSAN

A, E

Here’s why:

Question #60

An administrator has deployed the VMware Cloud Builder appliance in preparation for the deployment of a new VMware Cloud Foundation (VCF) environment. While performing the bring-up of the Management Domain, one of the tasks fails. The administrator is unable to identify the root cause of the failure and needs to generate a support bundle to send to VMware support. Which tool should the administrator use to generate the support bundle for the failed bring-up?

A. Generate support Bundle option on the vCenter server Management Interface B. Generate Support Bundle option on the vSphere Host Client Interface C. Supportability and Serviceability (SoS) Utility on the VMware Cloud Builder appliance D. Supportability and Serviceability (SoS) Utility on the SDDC Manager appliance

Here’s why:

Create Ubuntu Template on Proxmox

Wed, 10 Jul 2024 14:26:54 +0800

Ubuntu Server 24.04

# 1. SSH login Proxmox node
$ ssh root@192.168.61.11
# 2. 下載 Ubuntu Server 24.04 image
$ wget https://cloud-images.ubuntu.com/noble/20240702/noble-server-cloudimg-amd64.img
# 3. Customize the image
$ apt install -y libguestfs-tools
$ virt-customize --install net-tools,sudo -a noble-server-cloudimg-amd64.img
# 4. 建立虛擬主機, 並設定網路
$ vmid="9003"
$ qm create "$vmid" --name us2404 --memory 8192 --net0 virtio,bridge=vmbr0
# 5. 匯入虛擬主機硬碟檔, 設定使用 SCSI 介面連接 虛擬硬碟檔
$ qm importdisk "$vmid" noble-server-cloudimg-amd64.img local-lvm
$ qm set "$vmid" --scsihw virtio-scsi-pci --scsi0 local-lvm:vm-"$vmid"-disk-0
# 6. 加入 Cloud Init 功能
$ qm set "$vmid" --ide2 local-lvm:cloudinit
# 7. 撰寫 cloud-init User data 設定檔
$ mkdir -p /var/lib/vz/snippets/
$ nano /var/lib/vz/snippets/user"$vmid".yml
#cloud-config
hostname: alpine
package_update: true
packages:
- nano
write_files:
- path: /etc/resolv.conf
permissions: "0644"
owner: root
content: |
nameserver 8.8.8.8
users:
- name: bigred
no_user_group: true
lock_passwd: false
sudo: ALL=(ALL) NOPASSWD:ALL
plain_text_passwd: 'bigred'
homedir: /home/bigred
shell: /bin/bash
ssh_pwauth: True
# 8. 撰寫 cloud-init Network data 設定檔
$ nano /var/lib/vz/snippets/network"$vmid".yml
version: 1
config:
- type: physical
name: ens18
subnets:
- type: static
address: '192.168.61.31'
netmask: '255.255.255.0'
gateway: '192.168.61.2'
# 9. 設定 VM 套用設定檔
$ qm set "$vmid" --cicustom "user=local:snippets/user"$vmid".yml,network=local:snippets/network"$vmid".yml"
# 10. 改變硬碟大小
$ qm resize "$vmid" scsi0 100G
# 11. 設定開機硬碟
$ qm set "$vmid"--boot c --bootdisk scsi0
# 12. 設定成 Template
# qm template "$vmid"

Ubuntu Server 22.04

# 1. SSH login Proxmox node
$ ssh root@192.168.61.11
# 2. 下載 Ubuntu Server 22.04 image
$ wget https://cloud-images.ubuntu.com/jammy/20240705/jammy-server-cloudimg-amd64.img
# 3. Customize the image
$ apt install -y libguestfs-tools
$ virt-customize --install net-tools,sudo -a jammy-server-cloudimg-amd64.img
# 4. 建立虛擬主機, 並設定網路
$ vmid="9003"
$ qm create "$vmid" --name us2204 --memory 8192 --net0 virtio,bridge=vmbr0
# 5. 匯入虛擬主機硬碟檔, 設定使用 SCSI 介面連接 虛擬硬碟檔
$ qm importdisk "$vmid" jammy-server-cloudimg-amd64.img local-lvm
$ qm set "$vmid" --scsihw virtio-scsi-pci --scsi0 local-lvm:vm-"$vmid"-disk-0
# 6. 加入 Cloud Init 功能
$ qm set "$vmid" --ide2 local-lvm:cloudinit
# 7. 撰寫 cloud-init User data 設定檔
$ mkdir -p /var/lib/vz/snippets/
$ nano /var/lib/vz/snippets/user"$vmid".yml
#cloud-config
hostname: alpine
package_update: true
packages:
- nano
write_files:
- path: /etc/resolv.conf
permissions: "0644"
owner: root
content: |
nameserver 8.8.8.8
users:
- name: bigred
no_user_group: true
lock_passwd: false
sudo: ALL=(ALL) NOPASSWD:ALL
plain_text_passwd: 'bigred'
homedir: /home/bigred
shell: /bin/bash
ssh_pwauth: True
# 8. 撰寫 cloud-init Network data 設定檔
$ nano /var/lib/vz/snippets/network"$vmid".yml
version: 1
config:
- type: physical
name: ens18
subnets:
- type: static
address: '192.168.61.31'
netmask: '255.255.255.0'
gateway: '192.168.61.2'
# 9. 設定 VM 套用設定檔
$ qm set "$vmid" --cicustom "user=local:snippets/user"$vmid".yml,network=local:snippets/network"$vmid".yml"
# 10. 改變硬碟大小
$ qm resize "$vmid" scsi0 100G
# 11. 設定開機硬碟
$ qm set "$vmid"--boot c --bootdisk scsi0
# 12. 設定成 Template
# qm template "$vmid"

Remove and Rejoin a node on Proxmox Cluster

Sun, 23 Jun 2024 03:26:46 +0800

情境描述

如果直接把叢集中每一台 node 的 IP 或 Hostname 改掉，導致整個叢集的 node 之間互相失連，然後又遇到 /etc/pve/corosync.conf 完全無法修改，請根據以下步驟進行故障還原

1. 將叢集中所有其他的 node 從 corosync 中移除

# 1. SSH 連到第一台 node
$ ssh root@p1
# 2. Start your pve filesystem into local mode:
$ systemctl stop corosync pve-cluster
$ pmxcfs -l
# 3. Edit your corosync on a newly created file:
## 這裡只保留自己這台 node 的資訊
## totem.config_version 版號要 +1
$ nano correct_corosync.conf
logging {
debug: off
to_syslog: yes
}
nodelist {
node {
name: p1
nodeid: 1
quorum_votes: 1
ring0_addr: 192.168.37.50
}
}
quorum {
provider: corosync_votequorum
}
totem {
cluster_name: topgun
config_version: 8
interface {
linknumber: 0
}
ip_version: ipv4-6
link_mode: passive
secauth: on
version: 2
}
# 4. pull your new corosync configuration over your previous one
$ cp correct_corosync.conf /etc/pve/corosync.conf
$ cp correct_corosync.conf /etc/corosync/corosync.conf
# 5. Exit the local mode pve filesystem:
$ killall pmxcfs
# 6. Restart the services:
$ systemctl start pve-cluster corosync
# 7. 把叢集中其他節點的資料夾移除
$ rm -rf /etc/pve/nodes/{p2,p3,p4,p5}

2. Separate a Node Without Reinstalling

# 1. First, stop the corosync and pve-cluster services on the node:
$ systemctl stop pve-cluster corosync
# 2. Start the cluster file system again in local mode:
$ pmxcfs -l
[main] notice: resolved node name 'p2' to '192.168.37.51' for default node IP address
[main] notice: forcing local mode (although corosync.conf exists)
# 3. Delete the corosync configuration files:
$ rm /etc/pve/corosync.conf && \
rm -r /etc/corosync/*
# 4. You can now start the file system again as a normal service:
$ killall pmxcfs
$ systemctl start pve-cluster
# 5. The node is now separated from the cluster. You can deleted it from any remaining node of the cluster with:
$ pvecm delnode oldnode
# 6. If the command fails due to a loss of quorum in the remaining node, you can set the expected votes to 1 as a workaround:
$ pvecm expected 1
# 7. And then repeat the pvecm delnode command.
# 8. Now switch back to the separated node and delete all the remaining cluster files on it. This ensures that the node can be added to another cluster again without problems.
$ rm /var/lib/corosync/*
# 9. As the configuration files from the other nodes are still in the cluster file system, you may want to clean those up too. After making absolutely sure that you have the correct node name, you can simply remove the entire directory recursively from /etc/pve/nodes/NODENAME
## 先備份
$ cp -r /etc/pve/nodes /root
$ rm -rf /etc/pve/nodes/*
# 10. 確認網卡設定符合預期
$ cat /etc/network/interfaces
# 11. 確認 /etc/hosts 設定正確
$ cat /etc/hosts
# 12. 重啟 node
$ reboot

3. Rejoin node to proxmox cluster

# 1. 確認沒有 Failed 的服務
$ systemctl --failed
UNIT LOAD ACTIVE SUB DESCRIPTION
0 loaded units listed.

Join Node to Cluster via GUI

Log in to the web interface on an existing cluster node. Under Datacenter → Cluster, click the Join Information button at the top. Then, click on the button Copy Information. Alternatively, copy the string from the Information field manually.

Next, log in to the web interface on the node you want to add. Under Datacenter → Cluster, click on Join Cluster. Fill in the Information field with the Join Information text you copied earlier. Most settings required for joining the cluster will be filled out automatically. For security reasons, the cluster password has to be entered manually.

將虛擬機器的設定檔放回指定目錄區

$ cp /root/nodes/$(hostname)/qemu-server/* /etc/pve/nodes/$(hostname)/qemu-server

如果有換 Hostname 要複製舊的主機節點的設定檔到新的 Hostname 的目錄區底下，舉例 : Hsotname 從 p2 改成 p3，那麼命令就要改成
cp /root/nodes/p2/qemu-server/* /etc/pve/nodes/$(hostname)/qemu-server

Create VM with Terraform in Proxmox Cluster

Sat, 25 May 2024 16:42:18 +0800

1. 環境準備

1.1. 製作 Alpine Cloud-init VM Template

# 1. 下載 alpine cloud init 硬碟檔
$ wget https://dl-cdn.alpinelinux.org/alpine/v3.19/releases/cloud/nocloud_alpine-3.19.1-x86_64-bios-cloudinit-r0.qcow2
# 2. 安裝必要套件
$ apt update; apt install -y libguestfs-tools
# 3. 在 Alpine cloud init 硬碟檔安裝 qemu-guest-agent、bash、sudo 套件
$ virt-customize --install qemu-guest-agent,bash,sudo -a nocloud_alpine-3.19.1-x86_64-bios-cloudinit-r0.qcow2
# 4. 建立 Alpine 虛擬主機
$ vmid="9000"
$ qm create "$vmid" --name alp-cloudinit-v3.19.1 --memory 4096 --sockets 1 --cores 4 --net0 virtio,bridge=vmbr0
# 5. 匯入 alpine cloud init 硬碟檔,
$ qm importdisk "$vmid" nocloud_alpine-3.19.1-x86_64-bios-cloudinit-r0.qcow2 local-lvm
# 6. 設定使用 SCSI 介面連接 虛擬硬碟檔
$ qm set "$vmid" --scsihw virtio-scsi-pci --scsi0 local-lvm:vm-"$vmid"-disk-0
# 7. 改變硬碟大小
$ qm resize "$vmid" scsi0 50G
# 8. 加入 Cloud Init 功能
$ qm set "$vmid" --ide2 local-lvm:cloudinit
# 9. Set boot device
$ qm set "$vmid" --boot c --bootdisk scsi0
# 10. Enable serial device
$ qm set "$vmid" --serial0 socket --vga serial0
# 11. 撰寫 cloud-init User data 設定檔
$ mkdir -p /var/lib/vz/snippets/
$ nano /var/lib/vz/snippets/user"$vmid".yml
#cloud-config
hostname: alpine
package_update: true
packages:
- nano
write_files:
- path: /etc/resolv.conf
permissions: "0644"
owner: root
content: |
nameserver 8.8.8.8
users:
# - default
- name: bigred
# groups: bigred
no_user_group: true
lock_passwd: false
sudo: ALL=(ALL) NOPASSWD:ALL
plain_text_passwd: 'bigred'
homedir: /home/bigred
shell: /bin/bash
# ssh_authorized_keys:
# -
ssh_pwauth: True
#bootcmd:
# - echo -e "bigred\nbigred" | adduser -s /bin/bash bigred
# - echo -e "rancher\nrancher" | adduser -s /bin/bash rancher
# - echo -e "bigred\nbigred" | passwd root
# 12. 撰寫 cloud-init Network data 設定檔
$ nano /var/lib/vz/snippets/network"$vmid".yml
version: 1
config:
- type: physical
name: eth0
subnets:
- type: static
address: '192.168.61.21'
netmask: '255.255.255.0'
gateway: '192.168.61.2'
# 12. 設定 VM 套用設定檔
$ qm set "$vmid" --cicustom "user=local:snippets/user"$vmid".yml,network=local:snippets/network"$vmid".yml"
# 13. test
$ qm start "$vmid"
$ ssh bigred@192.168.61.21
Welcome to Alpine!
...
$ exit
$ qm stop "$vmid"
# 14. 將 VM 製作成 Template
$ qm template "$vmid"

1.2. 使用 API keys 認證

# 1. 創建一個名為 tfuser 的使用者
$ pveum user add terraform@pve
# 2. 創建一個名為 Terraform 的新角色，並賦予這個角色一系列與虛擬機管理和數據存儲管理相關的權限。這些權限允許用戶使用該角色來創建、配置、克隆、監控和管理虛擬機，以及管理數據存儲空間
$ pveum role add Terraform -privs "Datastore.AllocateSpace Datastore.Audit Pool.Allocate Sys.Audit Sys.Console Sys.Modify VM.Allocate VM.Audit VM.Clone VM.Config.CDROM VM.Config.Cloudinit VM.Config.CPU VM.Config.Disk VM.Config.HWType VM.Config.Memory VM.Config.Network VM.Config.Options VM.Migrate VM.Monitor VM.PowerMgmt SDN.Use"
# 3. 將 TerraformProv 角色分配給 tfuser，讓它擁有相應的權限。
$ pveum aclmod / -user terraform@pve -role Terraform
# 4. 產生 Terraform 要使用的 API Token
$ pveum user token add terraform@pve provider --privsep=0
┌──────────────┬──────────────────────────────────────┐
│ key │ value │
╞══════════════╪══════════════════════════════════════╡
│ full-tokenid │ terraform@pve!provider │
├──────────────┼──────────────────────────────────────┤
│ info │ {"privsep":"0"} │
├──────────────┼──────────────────────────────────────┤
│ value │ 1fb0b04e-5673-4d5c-b580-ae602c718d89 │
└──────────────┴──────────────────────────────────────┘

要儲存 Token Secrt，因為之後無法取得它。

1.3. 安裝 Terraform

$ apt-get update && apt-get install -y gnupg software-properties-common
$ wget -O- https://apt.releases.hashicorp.com/gpg | \
gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
$ echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] \
https://apt.releases.hashicorp.com $(lsb_release -cs) main" | \
tee /etc/apt/sources.list.d/hashicorp.list
$ apt update && apt install -y terraform

2. 建立 Terraform 設定檔

# 1. 建立儲存設定檔的工作目錄
$ mkdir terraform && cd terraform
# 2. 產生 backend config
$ cat <<EOF > backend.tf
terraform {
required_providers {
proxmox = {
source = "telmate/proxmox"
version = "3.0.1-rc2"
}
}
backend "local" {
}
}
EOF
# 3. 產生 provider config
$ cat <<EOF > provider.tf
provider "proxmox" {
pm_api_url = "https://192.168.61.11:8006/api2/json"
# api token id is in the form of: <username>@pam!<tokenId>
pm_api_token_id = "terraform@pve!provider"
# this is the full secret wrapped in quotes.
pm_api_token_secret = var.PROXMOX_API_SECRET
pm_tls_insecure = true
# debug log
# pm_log_enable = true
# pm_log_file = "terraform-plugin-proxmox.log"
# pm_debug = true
# pm_log_levels = {
# _default = "debug"
# _capturelog = ""
# }
}
EOF
# 4. 產生變數設定檔
$ cat <<EOF > vars.tf
variable "PROXMOX_API_SECRET" {
type = string
}
variable "ssh_key" {
default = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCa5vPXuVVZOQJw+mvQizMqOY+sJryVg/yPQHHq1NfLVZREbjUHvIMJcTVzEVpy6oO/v6pQ4wYbgUQqPmCP/Np/VvX6YBhdnhc99V8Jn9H99QhmXHVfrG3GgBQ5scupP5VUaPZ372gVsidu63uNyDvnSwZ0cAXUxvB3hANQBH0eJ13/zVKBkWVGQTd/eQJBQx5sxpwXW4sBcPaYdpt+Njtc31GqxwgSzTO9HAHp1aSDq4SA3VEjUNg7/ysW/MwuuslZBsz0wmi55nBa9dkHG1ao72c9viVWc+fz+nLjMRq6QDUxJd2TjQH4k7QexibDXCZydxac2tMRKRo3ukrdSs1TzjcnUdWtTnFM+wDEkWnObagkp9feRg5+7RWh+3qc2zq1/OCEMqGd6Gu17VUxWK4L3QEq+k13vgphuzQpziRiolh8ETvSM0EXajuHj5RoJQKAEhfURvFgH+qa5nUw8DTjtR2gfkevKXOJBTDT9GauTItZJi03fQhVAng1blP1d8OkD+yItHthiylBYW+UdMT/HCgc9JtkDzjtmfRH2HvDZnFJ1O46EX74wycEs7Lgyeu3vHIYC+AP7/XprT9jIT5UHo3MOJ4vZwP0OcFhqg0x+8ptjzWRtn1SubcNRsYlJlc59gqAPSyrsAtYL5FGohgGokxlRX8hDjUhcLmknFQhiw== root@pve"
}
variable "proxmox_host" {
default = "pve"
}
variable "template_name" {
default = "alp-cloudinit-v3.19.1"
}
EOF
# 5. 產生 Terraform VM 設定檔 main.tf
$ nano main.tf
resource "proxmox_vm_qemu" "ckad" {
count = 1
name = "alp-${count.index + 1}"
target_node = var.proxmox_host
vmid = "101${count.index + 1}"
clone = var.template_name
os_type = "cloud-init"
cpu = "host"
cores = 4
sockets = 1
memory = 4096
scsihw = "virtio-scsi-pci"
bootdisk = "scsi0"
disks {
scsi {
scsi0 {
disk {
size = "50G"
storage = "local-lvm"
}
}
}
ide {
ide3 {
cloudinit {
storage = "local-lvm"
}
}
}
}
network {
model = "virtio"
bridge = "vmbr0"
tag = -1
firewall = false
link_down = false
}
ipconfig0 = "ip=192.168.61.2${count.index + 1}/22,gw=192.168.61.2"
ciuser = "terraform"
sshkeys = <<EOF
${var.ssh_key}
EOF
}

2.1. 檢查目錄結構

$ tree ../terraform
../terraform
├── backend.tf
├── main.tf
├── provider.tf
└── vars.tf
1 directory, 4 files

3. 建立 VM

$ export PROXMOX_API_SECRET="1fb0b04e-5673-4d5c-b580-ae602c718d89"
# inicialise the terraform modules
$ terraform init --upgrade
# generate a terraform plan
$ terraform plan -out ckad.tfplan
# deploy the VMs
$ terraform apply -parallelism=1 -auto-approve ckad.tfplan
# cleanup
$ terraform plan -destroy -out ckad.tfplan
$ terraform apply "ckad.tfplan"

4. 參考文章

Cloud Init Guide - Terraform Docs

Shrink the VM Disk File on Proxmox

Sun, 19 May 2024 06:12:46 +0800

1. Get Volume ID

$ pvesm list local-lvm --vmid 534
Volid Format Type Size VMID
local-lvm:vm-534-cloudinit raw images 4194304 534
local-lvm:vm-534-disk-0 raw images 107374182400 534

2. Get the path to the disk

$ pvesm path local-lvm:vm-534-disk-0
/dev/pve/vm-534-disk-0

3. Create a Folder

$ mkdir ub2204

3. Shrink the Disk File

$ qemu-img convert -p -f raw -O qcow2 /dev/pve/vm-534-disk-0 /root/ub2204/ub2204.qcow2
(100.00/100%)

4. Check

$ ls -lh ub2204/
total 12G
-rw-r--r-- 1 root root 12G May 19 13:14 ub2204.qcow2
$ qm create 535 --name ubtest2 --memory 12288 --cores 4 --net0 virtio,bridge=vmbr0
$ qm importdisk 535 ~/ub2204/ub2204.qcow2 local-lvm
$ qm set 535 --scsihw virtio-scsi-pci --scsi0 local-lvm:vm-535-disk-0
$ qm set 535 --ide2 local-lvm:cloudinit
$ pvesm list local-lvm --vmid 535
Volid Format Type Size VMID
local-lvm:vm-535-cloudinit raw images 4194304 535
local-lvm:vm-535-disk-0 raw images 107374182400 535
$ qm set 535 --boot c --bootdisk scsi0
$ nano /var/lib/vz/snippets/user535.yml
#cloud-config
hostname: ub2204
package_update: true
packages:
- nano
write_files:
- path: /etc/resolv.conf
permissions: 0644
owner: root
content: |
nameserver 8.8.8.8
system_info:
default_user:
name: bigred
groups: bigred
lock_passwd: false
sudo: ALL=(ALL) NOPASSWD:ALL
plain_text_passwd: 'bigred'
homedir: /home/bigred
shell: /bin/bash
ssh_pwauth: True
# - bootcmd will run on every boot
#bootcmd:
# - echo "hi andy" > /home/andy/test
$ nano /var/lib/vz/snippets/network535.yml
version: 1
config:
- type: physical
name: ens18
subnets:
- type: static
address: '192.168.33.99'
netmask: '255.255.255.0'
gateway: '192.168.33.254'
$ qm set 535 --cicustom "user=local:snippets/user535.yml,network=local:snippets/network535.yml"
update VM 535: -cicustom user=local:snippets/user535.yml,network=local:snippets/network535.yml
$ qm start 535

清除 VMWare Player 虛擬主機清單

Mon, 18 Dec 2023 06:44:24 +0800

$ type "c:\Documents and Settings\%USERNAME%\Application Data\VMWare\preferences.ini" | findstr /v pref.mruVM > myvm.ini && copy /A /Y myvm.ini "c:\Documents and Settings\%USERNAME%\Application Data\VMWare\preferences.ini"

2023 Windows & VMware IaC 開發及使用手冊

Mon, 24 Apr 2023 14:18:10 +0800

緣由

要手動部屬多台虛擬主機時，有高機率造成 Human Error ，會產生大量時間成本

虛擬化技術中，一個資料夾代表一台電腦，要設定每一台虛擬主機的 hostname, IP 位址, 名稱解析檔…等，此時容易發生 Human Error。

當每一台虛擬主機手動做好以上設定時，要將每一台虛擬主機先壓縮，此時必須坐在電腦前面，手動將每一台虛擬主機都壓縮，壓縮完後，再手動將每一個壓縮檔，派送到對應的實體 Windows 電腦，再手動將每一台虛擬主機的壓縮檔解壓縮，最後再進到每一台虛擬主機的目錄，將虛擬主機啟動。

以上動作，如果沒有事先規劃好，壓縮檔的檔名，要放在哪個目錄，哪一台虛擬機要派送到哪一台實體 Windows 電腦，要解壓縮到哪一個目錄，當叢集需要用到的虛擬主機數量越大時，會造成管理上的不方便。

當以上動作完成，將已派送到對應 Windows 主機裡的虛擬主機都啟動後，又發現與當初規劃的規格不符，此時又要跳回第一步，會耗用大量時間成本。

認識 IaC

伺服器環境的手動配置給 IT 專業人員帶來了一些挑戰，例如，由於不斷增加的複雜性和對 IT 基礎架構的相關需求。

Infrastructure as Code，中文翻為 “基礎架構程式碼” ，提供了一種替代方法，將基礎架構的提供、配置與管理方式自動化。

Infrastructure 以實體比喻的話，就是機房裡的 Server

Iac 分為兩種方法，分別是 Declarative (聲明式) 和 Imperative (宣告式)

聲明式主要說明基礎系統架構要完成甚麼目標、需要的資源或任何狀態
宣告式定義了實現基礎系統架構所需的特定命令，並以正確的順序執行這些命令。

執行它的好處不僅可以降低 Human Error 、縮短建置系統的時間，還可以確保每次都建立出一模一樣的基礎架構。

舉例，在 CNT.2023.v4.6 的套件包裡面的 docker.bat, k8s.bat, dt.bat …等批次檔案，都能快速建立起，對應的基礎架構。

註解：IaC 技術幫我們從落地雲主機自動產生以及複製虛擬主機，並派送至多台不同的實體 Windows 10 主機

參考文章連結 :

Windows & VMware IaC 平台設計

平台目標

自動化建置與管理分散型叢集系統基礎架構

以 Kubernetes 架構範例 :

一台 Master 主機
三台 Worker 主機

以 Hadoop 架構範例 :

一台 Admin 機器
兩台 Master 機器
- M1 跑 Name Node, Secondary Name Node
- M2 跑 Resource Manager, Jobhistory Server
三台 Worker 機器
- 每一台機器都會跑 Data Node, Node Manager

重要操作步驟

設計叢集架構 (mactohost)
產生虛擬電腦資料夾
壓縮虛擬電腦資料夾
派送虛擬電腦資料夾
解壓縮所有虛擬主機壓縮檔
啟動與檢視所有虛擬主機
關閉所有虛擬主機
備份與還原所有虛擬主機
清除所有虛擬主機

實體電腦硬體規格

多台實體 Windows 10 電腦
記憶體至少 24 G
CPU: 4 C / 2 HT 以上
須為靜態 IPv4 的 IP 位址
每台實體 Windows 10 電腦會分別跑一台或多台不同的虛擬電腦

虛擬電腦硬體規格

Linux 作業系統
記憶體 : 依建置叢集需求設定
CPU : 依建置叢集需求設定
網路模式 : bridge

虛擬電腦網路設定

根據叢集的大小來規劃 IP 位址、 Subnetmask 以及 Default Gateway

開始建置 C1 Cluster (1M3W)

建置叢集系統前準備

關閉防火牆
確認實體 Windows 10 主機之間可用 SSH 連接 測試命令 :
```
> ssh bigred@<要派送的實體 Windows 10 電腦 的 IP 位址>
```
無法連線，請用滑鼠右鍵點選 Windows Powershell 圖示，選擇"以系統管理員身分執行"，並執行以下命令
- 檢查 OpenSSH 目前的狀態
```
PS > Get-WindowsCapability -Online | Where-Object Name -like 'OpenSSH*'
```
  如果 OpenSSH Server 沒有安裝，命令應返回以下輸出 :
```
Name : OpenSSH.Client~~~~0.0.1.0
State : Installed
Name : OpenSSH.Server~~~~0.0.1.0
State : NotPresent
```
- 安裝 OpenSSH Server 命令 :
```
PS > Add-WindowsCapability -Online -Name OpenSSH.Server~~~~0.0.1.0
```
  螢幕輸出 :
```
Path :
Online : True
RestartNeeded : False
```
- 啟動 sshd 服務命令
```
PS > Start-Service sshd
```
- 將 sshd 服務設為開機自動啟動
```
PS > Set-Service -Name sshd -StartupType 'Automatic'
```
- 連線要派送虛擬主機壓縮檔的實體 Windows 10 電腦的 IP 位址測試
```
> ssh bigred@<要派送的實體 Windows 10 電腦 的 IP 位址>
```
[註] Install OpenSSH Server on Windows 10 官網連結
將連接 Windows 實體主機使用者的密碼存入 passwd 檔案
```
> notepad %USERPROFILE%\CNT.2023.v4.6\C1-1M3W\passwd
```
將連接模板虛擬主機使用者的密碼存入 passwd_vm 檔案
```
> notepad %USERPROFILE%\CNT.2023.v4.6\C1-1M3W\passwd_vm
```
請執行 VMware Player 這軟體以下設定 步驟一 : Player > File > Preferences... 步驟二 : In the “Software updates” group > uncheck the “Check for software components as needed” option.
確認被遠端控制的實體 Windows 主機，開放讓 psexec 遠端連接權限用系統管理員身份打開 CMD，並執行以下命令
```
reg add HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\system /v LocalAccountTokenFilterPolicy /t REG_DWORD /d 1 /f
```
[註] 1. 機碼修改後，需重啟電腦 2. 如有安裝 F-Secure，請關閉 F-Secure 防火牆參考文章連結 : PsTools 於 Windows 10 的環境設定
確認 Windows 主機上當前的使用者家目錄下 ssh 的私鑰 %USERPROFILE%\.ssh\id_rsa 權限是否太開放，
如果在群組或其他使用者這個欄位下，有其他人，那麼請執行以下命令 :
```
> cd %USERPROFILE%\CNT.2023.v4.6\C1-1M3W && sshsetup.bat
```
螢幕輸出 :
```
SSH key Setup ok !
```

1. 設計 c1 叢集架構

設定 c1 叢集架構檔 - `mactohost`

在 Windows 中，打開 CMD ，執行以下命令

切換至工作目錄

> cd CNT.2023.v4.6\C1-3M3W

編輯 c1mactohost

> notepad c1mactohost

檔案內容 :

# Default Cluster
00:00 x1 none
00:01 x2 VMware
00:02 x3 VMware
00:03 a1 admin
00:04 m1 master
00:05 m2 master
00:06 m3 master
00:07 w1 worker
00:08 w2 worker
00:09 w3 worker
# K8S c1 Cluster ALP.K8S
10:f0 c1a1 admin
10:f1 c1m1 master c1m1.241.zip c1/alp.c1m1 120.96.143.170
10:f2 c1m2 master
10:f3 c1m3 master
10:f4 c1e1 etcd
10:f5 c1w1 worker c1w1.245.zip c1/alp.c1w1 120.96.143.170
10:f6 c1w2 worker c1w2.246.zip c1/alp.c1w2 120.96.143.171
10:f7 c1w3 worker c1w3.247.zip c1/alp.c1w3 120.96.143.171
10:f8 c1w4 worker
10:f9 c1w5 worker

欄位說明 (用空格做切割) :

先單獨看第 12 行內容，
```
# K8S c1 Cluster ALP.K8S
```
- 第三欄 c1 會與 CNT.2023.V4.6\C1-1A3M\c1mactohost 這個檔案的前兩個字 c1 比較，所以不能亂設定
- 第五欄 ALP.K8S 設定模板機的目錄名稱
  - 模板機必須放在 %USERPROFILE%\CNT.2023.v4.6\目錄下
第一欄為設定 VM 主機的 Mac Address 最後兩個位元 (byte)
- 用冒號分隔的格式 : 前半部是 c1 = 10, c2 = 20… 依此類推 後半部是 VM 的 IP 位址尾數換算為 16 進位
- 舉例 : 假設現在要設定的是 c1 叢集裡面 IP 位址尾數 .210 的 VM，依照上述規則，得到的答案 : 10:d2 Note: 1. 16 進位中的英文需設為小寫 2. 不能設為 : 10:xx ，會導致虛擬機器的 /etc/hosts 無法正常解析 ! 3. 在此檔案中的任何一台 VM 的 Mac Address 不可重複
第二欄為設定 VM 主機名稱
- 格式 : 叢集名稱 + mactohost 第三欄位的第一個字 + 序號(從 1 開始)
第三欄為各 VM 的角色扮演
第四欄為 VM 目錄壓縮檔名
- 格式 : VM 主機名稱 + VM IP 位址的尾數 + 壓縮檔的附檔名 (.zip)
第五欄為 VM 存放的目錄名稱
- 格式 : 叢集名稱/us.主機名稱 (us = Ubuntu Server 的縮寫)
- 格式 : 叢集名稱/alp.主機名稱 (alp = Alpine Linux 的縮寫)
第六欄為 VM 要分配到哪一台實體電腦的 IP 位址

2. 建立 c1 叢集所有虛擬主機

create.bat 程式概述：在落地雲主機的 C:\Users\bigred\CNT.2023.v4.6\C1-1M3W 目錄下，做出 c1 的資料夾，並在 c1 目錄下建立 4 台虛擬機 [重要] 根據實體電腦的記憶體來修改 vmcreate.bat 中 RSMMAX 變數直的大小

@echo off
REM 設定 Cluster Name
for /f "tokens=1" %%y in ('dir /B ^| findstr "mactohost"') do (
set mactohost=%%y
)
for /f "tokens=3 delims= " %%z in ('type %mactohost% ^| more +11 ^| findstr "#"') do (
set CN=%%z
)
REM 判斷 Cluster Name 有沒有設定錯誤
if NOT %mactohost:~0,2% == %CN% (
echo "Cluster Name Error !"
GOTO:eof
)
if exist %CN%\ (
echo "%CN% Folder existed !"
GOTO:eof
)
REM 設定 模板機所在目錄名稱 和 模板機的 vmx 檔案位置 變數
for /f "skip=1 tokens=5 delims= " %%a in ('type %mactohost% ^| findstr Cluster') do (
set vmfolder=%%a
for /f "tokens=1" %%g in ('dir /S /B ..\%%a\ ^| findstr vmx$') do (
set vmx=%%g
)
)
REM 將 c*mactohost 中 實體 Windows 主機的 IP 匯入 ip.txt 檔案中
if exist ip.txt (del ip.txt)
for /f "skip=12 tokens=6 delims= " %%y in (%mactohost%) do (
echo %%y >> ip.txt
)
REM 判斷要派送的主機 22 port 有沒有開, psexec 功能是否正常
setlocal EnableDelayedExpansion
for /f "tokens=1 delims= " %%h in ('type ip.txt ^| sort.exe /unique') do (
for /f "tokens=1" %%p in ('powershell -command "& {(New-Object System.Net.Sockets.TcpClient).ConnectAsync('%%h', 22).Wait(100)}"') do (
if %%p == False echo "%%h SSH Error" && GOTO:eof
)
REM 設定本機 IP 變數
for /f "delims=[] tokens=2" %%e in ('ping -4 -n 1 %ComputerName% ^| findstr [') do (
set HostIP=%%e
)
if not defined HostIP "echo !HostIP! should manual configuration in %~nx0"
if not !HostIP! == %%h (
psexec -accepteula -nobanner \\%%h -s "hostname" > nul 2> nul
)
if !ERRORLEVEL! NEQ 0 (
echo "%%h psexec Error" && GOTO:eof
)
)
del ip.txt 2> nul > nul
REM 判斷 VM 使用的 IP 目前有無被占用
for /f "skip=12 tokens=4,6 delims= " %%a in (%mactohost%) do (
for /f "tokens=1-3 delims=." %%x in ('echo %%b') do (
for /f "tokens=2 delims=." %%i in ('echo %%a') do (
ping -n 1 %%x.%%y.%%z.%%i | find "TTL" 2> nul >nul
if !ERRORLEVEL! EQU 0 echo "%%x.%%y.%%z.%%i is already being used." && GOTO:eof
)
)
)
)
REM 判斷 10 進位轉 16 進位有沒有算錯
for /f "tokens=1,2 delims=." %%k in ('type %mactohost% ^| findstr zip') do (
for /f "tokens=2 delims=:" %%s in ("%%k") do (
for /f "tokens=1" %%o in ("%%s") do (
set "h=%%o"
set "DEC=%%l"
cmd /C exit !DEC!
set "HEX=!=ExitCode!"
for /F "tokens=* delims=0" %%Z in ("!HEX!") do (
if /I !DEC! LSS 16 (
set "HEX=0%%Z"
for /F "delims=" %%r in ('powershell -command " '!HEX!'.ToLower() "') do (
set "SHEX=%%r"
if NOT "!h!" EQU "!SHEX!" echo The hexadecimal representation of !DEC! is !SHEX!, not !h!. && GOTO:eof
)
) else (
set "HEX=%%Z"
for /F "delims=" %%r in ('powershell -command " '!HEX!'.ToLower() "') do (
set "SHEX=%%r"
if NOT "!h!" EQU "!SHEX!" echo The hexadecimal representation of !DEC! is !SHEX!, not !h!. && GOTO:eof
)
)
)
)
)
)
REM 設定 VM 可用記憶體範圍
powershell -Command "(Get-CimInstance Win32_PhysicalMemory | Measure-Object -Property capacity -Sum).sum /8Mb" > max.txt
powershell -Command "(Get-CimInstance Win32_PhysicalMemory | Measure-Object -Property capacity -Sum).sum /8Mb" > min.txt
set /P MAX=<max.txt
set /P MIN=<min.txt
for /f "tokens=2 delims==" %%a in ('type %vmx% ^| findstr memsize') do (
set mem=%%a
)
set RSMMAX='memsize = \"%mem:~2,-1%\"', 'memsize = \"%MAX%\"'
set RSMMIN='memsize = \"%mem:~2,-1%\"', 'memsize = \"%MIN%\"'
set RSADR1='ethernet0.generatedAddressOffset = \"0\"'
set RSADR2='ethernet0.addressType = \"generated\"' , 'ethernet0.addressType = \"static\"'
set RSTYPE='ethernet0.connectionType = \"nat\"' , 'ethernet0.connectionType = \"bridge\"'
for %%i in (vmfolder vmx mem) do (
if not defined %%i (
echo %%i not defined && GOTO:eof
)
)
REM 上載 mactohosts 至 模板 虛擬主機
"C:\Program Files (x86)\VMware\VMware Player\vmrun" start "%vmx%" > nul
timeout /t 60 > nul
"C:\Program Files (x86)\VMware\VMware Player\vmrun" getGuestIPAddress "%vmx%" > admip.txt
set /P admip=<admip.txt
echo y | pscp -pwfile "passwd_vm" ".\%CN%mactohost" bigred@%admip%:/home/bigred/bin/mactohost 2> nul > nul
if %ERRORLEVEL% EQU 0 echo "%CN%mactohost scp ok"
echo y | pscp -pwfile "passwd_vm" bigred@%admip%:/home/bigred/.ssh/* %USERPROFILE%\.ssh\ 2> nul > nul
if %ERRORLEVEL% EQU 0 echo "%admip% .ssh/* scp ok"
del %USERPROFILE%\.ssh\known_hosts 2> nul
del %USERPROFILE%\.ssh\known_hosts.old 2> nul
type %vmx% | findstr Ubuntu > nul 2> nul
if %ERRORLEVEL% NEQ 0 (
ssh -o "ConnectTimeout=5" -o "StrictHostKeyChecking=no" bigred@%admip% "sudo apk update; sudo apk upgrade" > nul 2>nul
if !ERRORLEVEL! EQU 0 echo "System upgrade ok"
) else (
ssh -o "ConnectTimeout=5" -o "StrictHostKeyChecking=no" bigred@%admip% "sudo apt update; sudo apt -y upgrade; sudo apt -y autoremove" > nul 2>nul
if !ERRORLEVEL! EQU 0 echo "System upgrade ok"
)
"C:\Program Files (x86)\VMware\VMware Player\vmrun" stop "%vmx%" > nul
timeout /t 60 > nul
REM 設定模板機 displayName 變數
for /f "tokens=2 delims==" %%a in ('type %vmx% ^| findstr displayName') do (
set odn=%%a
)
REM 設定模板機 mac address 變數
for /f "tokens=2 delims==" %%a in ('type %vmx% ^| findstr "ethernet0.generatedAddress" ^| findstr ":"') do (
set vmac=%%a
)
for %%j in (odn vmac) do (
if not defined %%j (
echo %%j not defined && GOTO:eof
)
)
REM 建立 Cluster 的所有 VM
if exist %mactohost% (
if not exist %CN%\ (
mkdir %CN% > nul
if %ERRORLEVEL% EQU 0 (
for /f "tokens=1-6 delims= " %%a in ('findstr zip %mactohost%') do (
set x=%%~nxe
set RS='displayName = \"%odn:~2,-1%\"', 'displayName = \"!x!\"'
set y=%%a
set RSMAC='ethernet0.generatedAddress = \"%vmac:~2,-1%\"' , 'ethernet0.address = \"00:50:56:ab:!y!\"'
mkdir %CN%\!x! > nul
xcopy /E /Y ..\%vmfolder% %CN%\!x!\ > nul
for /f "tokens=1" %%p in ('dir %CN%\!x! /B /S ^| findstr vmx$') do set nvmx=%%p
powershell -Command "(gc !nvmx!) -replace "!RS!" -replace "!RSMAC!" -replace "%RSMMAX%" -replace "%RSTYPE%" -replace "%RSADR1%" -replace "%RSADR2%" | Out-File -Encoding "UTF8" vm.temp"
del /Q !nvmx!
copy /Y vm.temp !nvmx! > nul
if !ERRORLEVEL! EQU 0 echo "%CN%\!x!\ (%MAX%) ok"
)
del *.txt > nul
del vm.temp > nul
)
)
)

執行建立 c1 叢集命令

在執行命令前，請先確定是否已完成叢集系統準備

在 Windows 工作目錄 CNT.2023.v4.6\c1-1M3W 下，執行程式

> vmcreate.bat

螢幕輸出

"c1mactohost scp ok"
"192.168.61.131 .ssh/* scp ok"
"System upgrade ok"
"c1\alp.c1m1\ (4096) ok"
"c1\alp.c1w1\ (4096) ok"
"c1\alp.c1w2\ (4096) ok"
"c1\alp.c1w3\ (4096) ok"

[註] 如遇到需要輸入密碼，請參考以下連結 : Windows SSH: Permissions for ‘private-key’ are too open

3. 壓縮 c1 叢集所有虛擬主機

zip.bat 程式概述 : 此程式使用 7z 作為虛擬主機目錄的壓縮命令

@echo off
REM 設定 Cluster Name
for /f "tokens=1" %%y in ('dir /B ^| findstr "mactohost"') do (
set CN=%%y
)
REM 在 mactohost 中抓取壓縮時需要的字串
if exist %CN:~0,2%zip.txt (del %CN:~0,2%zip.txt > nul)
for /f "tokens=3,* delims= " %%a in (%CN:~0,2%mactohost) do echo %%b | findstr zip >> %CN:~0,2%zip.txt
REM 透過 For 迴圈，逐行代變數壓縮 VM 所在的資料夾
setlocal EnableDelayedExpansion
for /f "tokens=1,2 delims= " %%a in (%CN:~0,2%zip.txt) do (
7z\7z -mx1 a %CN:~0,2%\%%a %%b > nul
if !ERRORLEVEL! EQU 0 (echo "%%a ok")
)
del %CN:~0,2%zip.txt > nul

執行壓縮程式

> vmzip.bat

螢幕輸出

"c1m1.141.zip ok"
"c1w1.145.zip ok"
"c1w2.146.zip ok"
"c1w3.147.zip ok"

4. 派送 c1 叢集所有虛擬主機

scp.bat 概述 : 使用 pscp.exe 命令，將 c1 叢集每一台虛擬主機的壓縮檔派送到 mactohost 中設定對應的各台實體 Windows 主機

@echo off
REM 判斷叢集名稱
for /f "tokens=1" %%y in ('dir /B ^| findstr "mactohost"') do (
set CN=%%y
)
REM 設定本機 IP 變數
for /f "delims=[] tokens=2" %%e in ('ping -4 -n 1 %ComputerName% ^| findstr [') do (
set HostIP=%%e
)
REM 抓取在 d*mactohost 中，每一台虛擬主機的相關資訊，並將資料逐行匯入 d*scp.txt
if exist %CN:~0,2%scp.txt (del %CN:~0,2%scp.txt > nul)
for /f "tokens=1* delims= " %%a in (%CN:~0,2%mactohost) do echo %%b | findstr zip >> %CN:~0,2%scp.txt
REM 將每一台虛擬主機的壓縮檔透過 pscp 命令，派送至對應的實體 Windows 主機的使用者家目錄
setlocal EnableDelayedExpansion
for /f "tokens=3,5 delims= " %%a in (%CN:~0,2%scp.txt) do (
if %%b == %HostIP% (
xcopy /Y %CN:~0,2%\%%a %USERPROFILE%\ > nul
if !ERRORLEVEL! EQU 0 (
echo "%%a to %%b ok"
) else (
echo "%%a to %%b failed"
)
) else (
echo y | pscp -scp -q -pwfile passwd "%CN:~0,2%\%%a" %USERNAME%@%%b:"%%a" 2> nul > nul
if !ERRORLEVEL! EQU 0 (
echo "%%a to %%b ok"
) else (
echo "%%a to %%b failed"
)
)
)
del %CN:~0,2%scp.txt > nul

執行程式

> vmscp.bat

螢幕輸出

"c1m1.141.zip to 120.96.143.152 ok"
"c1w1.145.zip to 120.96.143.152 ok"
"c1w2.146.zip to 120.96.143.153 ok"
"c1w3.147.zip to 120.96.143.153 ok"

5. 解壓縮所有虛擬主機壓縮檔至落地雲主機

unzip.bat 程式概述 : 本程式在本機使用 7z 將虛擬主機的壓縮檔解壓縮，遠端的主機使用 Windows Powershell 的 Expand-Archive 命令

@echo off
REM 判斷叢集名稱
for /f "tokens=1" %%y in ('dir /B ^| findstr "mactohost"') do (
set CN=%%y
)
REM 設定本機 IP 變數
for /f "delims=[] tokens=2" %%a in ('ping -4 -n 1 %ComputerName% ^| findstr [') do (
set HostIP=%%a
)
REM 解壓縮在本地或遠端的虛擬主機壓縮檔
for /f "skip=12 tokens=4-6 delims= " %%a in (%CN:~0,2%mactohost) do (
if %%c == %HostIP% (
setlocal EnableDelayedExpansion
"%ProgramFiles%\7-Zip\7z" "x" "C:\Users\%USERNAME%\%%a" "-oC:\Users\%USERNAME%" > nul 2> nul
if !ERRORLEVEL! EQU 0 (
echo "%%c %CN:~0,2%\%%~nxb unzip ok !"
) else (
echo "%%c %CN:~0,2%\%%~nxb unzip Failed !"
)
setlocal DisableDelayedExpansion
) else (
for /f "tokens=1 delims= " %%i in ('PsExec.exe -accepteula -nobanner \\%%c -s "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" "-Command" "Test-Path" "-Path" "C:\Users\%USERNAME%\%CN:~0,2%\%%~nxb" ^2^>nul') do (
if %%i == True (
psexec -accepteula -nobanner \\%%c -s "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" "-Command" "Remove-Item" "C:\Users\%USERNAME%\%CN:~0,2%\%%~nxb" "-Recurse" 2> nul
psexec -accepteula -nobanner \\%%c -s "powershell" "-Command" "Expand-Archive" "-LiteralPath" "C:\Users\%USERNAME%\%%a" "-DestinationPath" "C:\Users\%USERNAME%" 2> nul
) else (
psexec -accepteula -nobanner \\%%c -s "powershell" "-Command" "Expand-Archive" "-LiteralPath" "C:\Users\%USERNAME%\%%a" "-DestinationPath" "C:\Users\%USERNAME%" 2> nul
)
)
for /f "tokens=1 delims= " %%i in ('PsExec.exe -accepteula -nobanner \\%%c -s "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" "-Command" "Test-Path" "-Path" "C:\Users\%USERNAME%\%CN:~0,2%\%%~nxb" ^2^>nul') do (
if %%i == True (
echo "%%c %CN:~0,2%\%%~nxb unzip ok !"
) else (
echo "%%c %CN:~0,2%\%%~nxb unzip Failed !"
)
)
)
)

[註] %%~nxb，在範例中 %%b 為包含目錄的檔案名稱 : c1/alp.c1w1， %%~nb - 顯示 alp (%%b 的檔名) %%~xb - 顯示 .c1w1 ( %%b 的副檔名，包含 .)

執行解壓縮程式

> vmunzip.bat

螢幕輸出

"120.96.143.152 c1\alp.c1m1 unzip ok !"
"120.96.143.152 c1\alp.c1w1 unzip ok !"
"120.96.143.153 c1\alp.c1w2 unzip ok !"
"120.96.143.153 c1\alp.c1w3 unzip ok !"

6. 啟動與檢視 c1 叢集所有虛擬主機

start.bat 程式概述 : 使用 VMware 的 vmrun 命令，配合 psexec 命令將遠端 Windows 實體主機中的虛擬主機啟動

@echo off
REM 判斷叢集名稱
for /f "tokens=1" %%y in ('dir /B ^| findstr "mactohost"') do (
set CN=%%y
)
REM 設定本機 IP 變數
for /f "delims=[] tokens=2" %%a in ('ping -4 -n 1 %ComputerName% ^| findstr [') do (
set HostIP=%%a
)
setlocal EnableDelayedExpansion
for /f "tokens=6 delims= " %%a in (%CN:~0,2%mactohost) do (
for /f "tokens=1" %%k in ('dir %USERPROFILE%\%CN:~0,2% /S /B ^| findstr vmx$') do (
for /f "tokens=5,* delims=\" %%l in ("%%k") do (
set hostvmx=%%m
)
)
for /f "tokens=1" %%x in ('PsExec.exe -accepteula -nobanner \\%%a -s "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" "-Command" "Get-ChildItem" "-Path" "%USERPROFILE%\%CN:~0,2%" "-Name" "-Recurse" "-Force" ^2^>nul ^| findstr vmx$') do (
for /f "tokens=1,* delims=\" %%y in ("%%x") do (
set remotevmx=%%z
)
)
)
REM 啟動每一台虛擬主機
for /f "skip=12 tokens=4-6 delims= " %%a in (%CN:~0,2%mactohost) do (
if %%c == %HostIP% (
"C:\Program Files (x86)\VMware\VMware Player\vmrun" start "C:\Users\%USERNAME%\%CN:~0,2%\%%~nxb\%hostvmx%" nogui 2> nul > nul
if !ERRORLEVEL! EQU 0 (
echo "%CN:~0,2%\%%~nxb started"
) else (
echo "%CN:~0,2%\%%~nxb failed"
)
timeout /t 5 > nul
) else (
psexec \\%%c -accepteula -nobanner -s "C:\Program Files (x86)\VMware\VMware Player\vmrun" "start" "C:\Users\%USERNAME%\%CN:~0,2%\%%~nxb\%remotevmx%" "nogui" 2> nul > nul
if !ERRORLEVEL! EQU 0 (
echo "%CN:~0,2%\%%~nxb started"
) else (
echo "%CN:~0,2%\%%~nxb failed"
)
timeout /t 5 > nul
)
)

1. 執行啟動 c1 叢集所有虛擬主機命令

> vmstart.bat

螢幕輸出

"c1\alp.c1m1 started"
"c1\alp.c1w1 started"
"c1\alp.c1w2 started"
"c1\alp.c1w3 started"

2. 在 Windows 檢視已開啟的虛擬電腦

list.bat 程式概述 : 透過 ssh 遠端對虛擬主機執行命令，並檢視虛擬主機的狀態

@echo off
REM 判斷叢集名稱
for /f "tokens=1" %%y in ('dir /B ^| findstr "mactohost"') do (
set CN=%%y
)
REM 先用 ping 命令，檢測每一台虛擬主機網路，有回應則用 ssh 命令遠端對虛擬主機執行命令
for /f "skip=12 tokens=4,6 delims= " %%a in (%CN:~0,2%mactohost) do (
for /f "tokens=1-3 delims=." %%x in ('echo %%b') do (
for /f "tokens=2 delims=." %%i in ('echo %%a') do (
ping -n 2 "%%x.%%y.%%z.%%i" | find "TTL" 2> nul > nul
if errorlevel 1 (
echo "%%x.%%y.%%z.%%i not running"
) else (
ssh -o "ConnectTimeout=5" -o "StrictHostKeyChecking=no" bigred@"%%x.%%y.%%z.%%i" "n=$(hostname;hostname -i); echo $n running" 2> nul
)
)
)
)

執行檢測命令

> vmlist.bat

螢幕輸出 :

c1m1 120.96.143.141 running
c1w1 120.96.143.145 running
c1w2 120.96.143.146 running
c1w3 120.96.143.147 running

[註] 如將 VM 開機後，遇到類似以下錯誤訊息

`8f: value too great for base (error token is "8f")`

請將 /home/bigred/bin/chnameip 程式中原本的 51 行命令 ip=$((16#$v)) ，修改為 ip=$(printf "%d\n" 0x"$v")

7. 關閉 c1 叢集所有虛擬主機

stop.bat 程式概述 : 利用 ssh 遠端對虛擬主機執行 sudo poweroff 的命令，將虛擬主機關閉

@echo off
REM 判斷叢集名稱
for /f "tokens=1" %%y in ('dir /B ^| findstr "mactohost"') do (
set CN=%%y
)
REM 利用 ssh 遠端對每一台虛擬主機執行關機命令
setlocal EnableDelayedExpansion
for /f "skip=12 tokens=4,6 delims= " %%a in (%CN:~0,2%mactohost) do (
for /f "tokens=1-3 delims=." %%x in ('echo %%b') do (
for /f "tokens=2 delims=." %%i in ('echo %%a') do (
ping -n 2 "%%x.%%y.%%z.%%i" | find "TTL" 2> nul > nul
if errorlevel 1 (
echo "%%x.%%y.%%z.%%i not running"
) else (
ssh -o "ConnectTimeout=5" -o "StrictHostKeyChecking=no" bigred@"%%x.%%y.%%z.%%i" "sudo poweroff; exit" 2> nul > nul
if !ERRORLEVEL! EQU 0 (
echo "%%x.%%y.%%z.%%i stoped"
) else (
if !ERRORLEVEL! EQU -1 echo "%%x.%%y.%%z.%%i stoped"
)
)
)
)
)

執行關機 c1 叢集所有虛擬主機命令

> vmstop.bat

螢幕輸出

"120.96.143.141 stoped"
"120.96.143.145 stoped"
"120.96.143.146 stoped"
"120.96.143.147 stoped"

8. 備份與還原 c1 叢集所有虛擬主機

backup.bat 程式概述 : 如果虛擬主機在本機，會透過 7z 壓縮，如果在遠端，會配合 psexec 命令，壓縮遠端的虛擬主機

@echo off
REM 判斷叢集名稱
for /f "tokens=1" %%y in ('dir /B ^| findstr "mactohost"') do (
set CN=%%y
)
REM 設定本機 IP 變數
for /f "delims=[] tokens=2" %%e in ('ping -4 -n 1 %ComputerName% ^| findstr [') do (
set HostIP=%%e
)
REM 先用 ping 命令，檢測每一台虛擬主機是否已關機
setlocal EnableDelayedExpansion
for /f "skip=12 tokens=4,6 delims= " %%a in (%CN:~0,2%mactohost) do (
for /f "tokens=1-3 delims=." %%x in ('echo %%b') do (
for /f "tokens=1,2 delims=." %%i in ('echo %%a') do (
ping -n 2 "%%x.%%y.%%z.%%j" | find "TTL" 2> nul > nul
if !ERRORLEVEL! == 0 (
echo "Please Execute vmstop.bat" && GOTO:eof
)
)
)
)
for /f "skip=12 tokens=4-6 delims= " %%a in (%CN:~0,2%mactohost) do (
if %%c == %HostIP% (
if not exist "%USERPROFILE%\7-Zip" (
xcopy /E /Y ".\7z" "%USERPROFILE%\7-Zip\" > nul
)
"%USERPROFILE%\7-Zip\7z" -mx1 a "C:\Users\%USERNAME%\%%a.bak" "C:\Users\%USERNAME%\%%b" 2> nul > nul
if !ERRORLEVEL! EQU 0 (
echo "%%c %%a.bak backup ok !"
) else (
echo "%%c %%a.bak backup failed !"
)
) else (
for /f "tokens=1 delims= " %%i in ('PsExec.exe -accepteula -nobanner \\%%c -s "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" "-Command" "Test-Path" "-Path" "%USERPROFILE%\7-Zip" ^2^>nul') do (
if %%i == False (
echo y | pscp -scp -pwfile passwd -q -r ".\7z" %USERNAME%@%%c:"%USERPROFILE%\7-Zip\"
)
)
for /f "tokens=1 delims= " %%i in ('PsExec.exe -accepteula -nobanner \\%%c -s "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" "-Command" "Test-Path" "-Path" "C:\Users\%USERNAME%\%CN:~0,2%\%%~nxb" ^2^>nul') do (
if %%i == True (
psexec -accepteula -nobanner \\%%c -s "%USERPROFILE%\7-Zip\7z" "-mx1" "a" "C:\Users\%USERNAME%\%%a.bak" "C:\Users\%USERNAME%\%%b" 2> nul > nul
if !ERRORLEVEL! EQU 0 (
echo "%%c %%a.bak backup ok !"
) else (
echo "%%c %%a.bak backup failed !"
)
) else (
echo "C:\Users\%USERNAME%\%CN:~0,2%\%%~nxb VM folder not found !"
GOTO:eof
)
)
)
)

執行備份 c1 叢集所有虛擬主機命令

> vmbackup.bat

螢幕輸出 :

"120.96.143.152 c1m1.141.zip.bak backup ok "
"120.96.143.152 c1w1.145.zip.bak backup ok "
"120.96.143.153 c1w2.146.zip.bak backup ok "
"120.96.143.153 c1w3.147.zip.bak backup ok "

執行失敗的螢幕輸出

"Please Execute vmstop.bat"

可能原因 : 只要有一台 c1 cluster VM 沒關，就會噴出此錯誤訊息

[註] 新的備份檔會直接覆蓋掉舊的備份檔

執行還原 c1 叢集所有虛擬主機命令

restore.bat 程式概述 : 如果虛擬主機在本機，會透過 7z 解壓縮，如果在遠端，會配合 psexec 命令，解壓縮遠端的虛擬主機

@echo off
REM 判斷叢集名稱
for /f "tokens=1" %%y in ('dir /B ^| findstr "mactohost"') do (
set CN=%%y
)
REM 設定本機 IP 變數
for /f "delims=[] tokens=2" %%e in ('ping -4 -n 1 %ComputerName% ^| findstr [') do (
set HostIP=%%e
)
REM 先用 ping 命令，檢測每一台虛擬主機是否已關機
setlocal EnableDelayedExpansion
for /f "skip=12 tokens=4,6 delims= " %%a in (%CN:~0,2%mactohost) do (
for /f "tokens=1-3 delims=." %%x in ('echo %%b') do (
for /f "tokens=1,2 delims=." %%i in ('echo %%a') do (
ping -n 2 "%%x.%%y.%%z.%%j" | find "TTL" 2> nul > nul
if !ERRORLEVEL! == 0 (
echo "Please Execute vmstop.bat" && GOTO:eof
)
)
)
)
for /f "skip=12 tokens=4-6 delims= " %%a in (%CN:~0,2%mactohost) do (
if %%c == %HostIP% (
if exist "C:\Users\%USERNAME%\%CN:~0,2%\%%~nxb" (
RD /S /Q "C:\Users\%USERNAME%\%CN:~0,2%\%%~nxb" > nul 2> nul
"%USERPROFILE%\7-Zip\7z" x "C:\Users\%USERNAME%\%%a.bak" "-oC:\Users\%USERNAME%\%CN:~0,2%" > nul 2> nul
) else (
"%USERPROFILE%\7-Zip\7z" x "C:\Users\%USERNAME%\%%a.bak" "-oC:\Users\%USERNAME%\%CN:~0,2%" > nul 2> nul
)
if exist "C:\Users\%USERNAME%\%CN:~0,2%\%%~nxb" (
echo "%%c %CN:~0,2%\%%~nxb restore ok !"
) else (
echo "%%c %CN:~0,2%\%%~nxb restore Failed !"
)
) else (
for /f "tokens=1 delims= " %%i in ('PsExec.exe -accepteula -nobanner \\%%c -s "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" "-Command" "Test-Path" "-Path" "C:\Users\%USERNAME%\%CN:~0,2%\%%~nxb" ^2^>nul') do (
if %%i == True (
psexec -accepteula -nobanner \\%%c -s "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" "-Command" "Remove-Item" "C:\Users\%USERNAME%\%CN:~0,2%\%%~nxb" "-Recurse" 2> nul
psexec -accepteula -nobanner \\%%c -s "%USERPROFILE%\7-Zip\7z" "x" "C:\Users\%USERNAME%\%%a.bak" "-oC:\Users\%USERNAME%\%CN:~0,2%" > nul 2> nul
) else (
psexec -accepteula -nobanner \\%%c -s "%USERPROFILE%\7-Zip\7z" "x" "C:\Users\%USERNAME%\%%a.bak" "-oC:\Users\%USERNAME%\%CN:~0,2%" > nul 2> nul
)
)
for /f "tokens=1 delims= " %%i in ('PsExec.exe -accepteula -nobanner \\%%c -s "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" "-Command" "Test-Path" "-Path" "C:\Users\%USERNAME%\%CN:~0,2%\%%~nxb" ^2^>nul') do (
if %%i == True (
echo "%%c %CN:~0,2%\%%~nxb restore ok !"
) else (
echo "%%c %CN:~0,2%\%%~nxb restore Failed !"
)
)
)
)

> vmrestore.bat

螢幕輸出 :

"120.96.143.152 c1\alp.c1m1 restore ok "
"120.96.143.152 c1\alp.c1w1 restore ok "
"120.96.143.153 c1\alp.c1w2 restore ok "
"120.96.143.153 c1\alp.c1w3 restore ok "

執行失敗的螢幕輸出

"Please Execute vmstop.bat"

可能原因 : 只要有一台 c1 cluster VM 沒關，就會噴出此錯誤訊息

9. 清除 c1 叢集所有虛擬主機

del.bat 程式概述 : 如果虛擬主機在本地，會利用 Windows Powershell 的 Remove-Item 命令，刪除 c1 叢集所有虛擬主機、壓縮檔、本地的 c1 目錄，如果虛擬主機在遠端，會利用 Remove-Item 命令配合 psexec 命令刪除

@echo off
REM 設定 Cluster Name
for /f "tokens=1" %%y in ('dir /B ^| findstr "mactohost"') do (
set CN=%%y
)
REM 設定本機 IP 變數
for /f "delims=[] tokens=2" %%a in ('ping -4 -n 1 %ComputerName% ^| findstr [') do (
set HostIP=%%a
)
REM 判斷是否已備份
for /f "tokens=4-6 delims= " %%a in ('type %CN:~0,2%mactohost ^| findstr zip') do (
if %%c == %HostIP% (
if not exist %HOMEPATH%\%%a.bak (
echo "Please Execute vmbackup.bat first" && GOTO:eof
)
) else (
for /f "tokens=1 delims= " %%i in ('PsExec.exe -accepteula -nobanner \\%%c -s "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" "-Command" "Test-Path" "-Path" "C:\Users\%USERNAME%\%%a.bak" ^2^>nul') do (
if %%i == False (
echo "Please Execute vmbackup.bat first" && GOTO:eof
)
)
)
)
REM 先用 ping 命令，檢測每一台虛擬主機是否已關機
setlocal EnableDelayedExpansion
for /f "skip=12 tokens=4,6 delims= " %%a in (%CN:~0,2%mactohost) do (
for /f "tokens=1-3 delims=." %%x in ('echo %%b') do (
for /f "tokens=1,2 delims=." %%i in ('echo %%a') do (
ping -n 2 "%%x.%%y.%%z.%%j" | find "TTL" 2> nul > nul
if !ERRORLEVEL! == 0 (
echo "Please Execute vmstop.bat" && GOTO:eof
)
)
)
)
REM 將 d*mactohost 中 實體 Windows 主機的 IP 匯入 ip.txt 檔案中
if exist ip.txt (del ip.txt)
for /f "skip=12 tokens=6 delims= " %%y in (%CN:~0,2%mactohost) do echo %%y >> ip.txt
REM 刪除 ip.txt 檔案中每一台實體 Windows 電腦 使用者家目錄中以叢集為名稱的目錄
setlocal EnableDelayedExpansion
for /f "tokens=1 delims= " %%x in ('type ip.txt ^| sort.exe /unique') do (
if %%x == %HostIP% (
"C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -Command Remove-Item C:\Users\%USERNAME%\%CN:~0,2% -Recurse 2> nul
if !ERRORLEVEL! EQU 0 (
echo "%%x %CN:~0,2% folder deleted !"
) else (
echo "%%x %CN:~0,2% folder does not exist !"
)
) else (
psexec -accepteula -nobanner \\%%x -s "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" "-Command" "Remove-Item" "C:\Users\%USERNAME%\%CN:~0,2%" "-Recurse" 2> nul
if !ERRORLEVEL! EQU 0 (
echo "%%x %CN:~0,2% folder deleted !"
) else (
echo "%%x %CN:~0,2% folder does not exist !"
)
)
)
REM 刪除在本地或遠端的虛擬主機壓縮檔
for /f "skip=12 tokens=4-6 delims= " %%a in (%CN:~0,2%mactohost) do (
if %%c == %HostIP% (
"C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -Command Remove-Item C:\Users\%USERNAME%\%%a 2> nul
if !ERRORLEVEL! EQU 0 (
echo "%%c %%a deleted !"
) else (
echo "%%c %%a does not exist !"
)
) else (
psexec -accepteula -nobanner \\%%c -s "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" "-Command" "Remove-Item" "C:\Users\%USERNAME%\%%a" 2> nul
if !ERRORLEVEL! EQU 0 (
echo "%%c %%a deleted !"
) else (
echo "%%c %%a does not exist !"
)
)
)
del ip.txt 2> nul > nul
REM 刪除執行程式所在的目錄 (例如 : %USERPROFILE%\CNT.2023.v4.6\D9-1A2M3W\) 中的 d9 目錄
"C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -Command Remove-Item %CN:~0,2% -Recurse 2> nul
if %ERRORLEVEL% == 0 (
echo "%HostIP% %CD%\%CN:~0,2% folder deleted !"
) else (
echo "%HostIP% %CD%\%CN:~0,2% folder does not exist !"
)

執行清除 c1 叢集所有虛擬主機命令

> vmdel.bat

螢幕輸出 :

"120.96.143.152 c1 folder deleted "
"120.96.143.153 c1 folder deleted "
"120.96.143.152 c1m1.141.zip deleted "
"120.96.143.152 c1w1.145.zip deleted "
"120.96.143.153 c1w2.146.zip deleted "
"120.96.143.153 c1w3.147.zip deleted "
"120.96.143.170 C:\Users\bigred\CNT.2023.v4.6\C1-1M3W\c1 folder deleted "

執行失敗的螢幕輸出

"Please Execute vmbackup.bat first"

可能原因 : 程式一開始會檢查每一台 VM 對應的實體 Windows 主機的使用者家目錄，有無 *.bak 的備份檔，只要少一個備份檔，此程式就會噴出此錯誤訊息

[註] 會保留在本地或遠端實體主機的虛擬主機備份 (*.bak) 檔

IaC 平臺開發關鍵技術

技術重點 1 - Windows batch script - 透過 `FOR` 命令可以將文字檔中的欄位資料取出

:star: 首先要注意 ! 如果要在批次程式中使用 FOR 命令，請指定 %%variable，而不要指定%variable。變數名稱有大小寫的區分，所以 %i 不同於 %I。

逐行讀取文字檔的 `for /F` 迴圈

/f ，可以對 for 迴圈傳入的檔案、字串或命令，進行解析，以下為讀取檔案的範例 :

for /f "tokens=3,* delims= " %%a in (c1mactohost) do echo %%b

概述：逐行讀取 k1mactohost 檔案的內容，然後依照分隔符號 (空格) 將檔案內容的值分別設為指定的變數，再將指定的變數內容輸出在螢幕上。
tokens=x,y,m-n 用來決定一次要取出幾個欄位，第一個欄位會存放在第一個自動變數，第二個欄位會存放在第二個自動變數裡，依此類推。在範例中為tokens=3,*，代表將第三個欄位的值存放在變數 a ，其他剩餘欄位的值會放在變數 b 註: x, x, m, n 都為數字，只是 m-n 是一種特別的表示法，代表一個解析欄位的範圍。例如 tokens=2-4 就代表取得第2、第 3 與第 4 個欄位的值並分別塞指定的變數裡
delims=xxx 用來決定欄位的分隔符號，預設為空格與 TAB 符號，並可自訂多個符號。在範例中，將欄位的分隔符號設為空格註: xxx 代表多個字元，如果要同時設定小數點 . 與逗號 , 就要輸入delims=.,

練習

啟動一個文字檔 tip.txt

> notepad tip.txt

內容是

10:6e d8a5 120.96.143.22
10:6f d8a6 120.96.143.22

寫一隻程式（tip.bat）用冒號分割顯示以下的結果：

6e
6f

答案：

先編輯一個批次檔

$ notepad tip.bat

內容是

@echo off
for /f "tokens=1 delims= " %%a in (tip.txt) do echo %%a

將 a 變數匯入一個檔案

@echo off
for /f "tokens=1 delims= " %%a in (tip.txt) do echo %%a >> t1.txt
for /f "tokens=2 delims=:" %%b in (t1.txt) do echo %%b

執行批次檔

$ tip.bat

螢幕輸出

6e
6f

另一種答案，雙迴圈

@echo off
for /f "tokens=1 delims= " %%a in (tip.txt) do (
for /f "tokens=1,2 delims=:" %%b in ('echo %%a') do echo %%b %%c
)

甚麼是 `@echo off` ?

認識 echo on

在 Windows batch script 中，預設是 echo on，當 Windows 電腦在執行批次程式中的每一行命令時，會先顯示其解析後的結果，才會再顯示執行後的結果

範例 :

編輯一個 test.bat 批次檔

> notepad test.bat

檔案內容

set a=b
echo %a%

執行程式

> test.bat

螢幕輸出

>set a=b
>echo b
b

可以看到它在執行程式中的每一行命令時，會先顯示其解析後的結果，才會再顯示執行後的結果

認識 @echo off

@echo off 可以分為兩個部分，最前面的 @ 和後面的echo off

@ 是針對特定命令，讓它不要顯示被解析後的結果，直接顯示命令執行後的結果
echo off 通常會被加在 Windows 批次檔的最上面，它的作用是，程式在執行時，只顯示命令執行結果。

所以 @echo off ，它會不顯示自己被解析後的結果，且程式的每一行命令在執行時，只顯示命令執行結果。

延續上一個範例 :

編輯 test.bat 批次檔

> notepad test.bat

檔案內容的最上面新增 @echo off

@echo off
set a=b
echo %a%

執行程式

> test.bat

螢幕輸出

甚麼是 setlocal EnableDelayedExpansion ?

在了解甚麼是 setlocal EnableDelayedExpansion 之前，先來看看以下一段批次程式的內容

set VAR=before
if "%VAR%" == "before" (
set VAR=after
if "%VAR%" == "after" @echo "If you see this, it worked"
)

看完以後您覺得執行這隻程式的結果會顯示 "If you see this, it worked" 這段字串嗎?

來看看程式執行結果

> set VAR=before
> if "before" == "before" (
set VAR=after
if "before" == "after"
)

在預設情況下，每行命令被執行之前，變數擴展成值，只會發生一次，即使你在 if 的括號之中，把它分成兩行。

通過 SETLOCAL EnableDelayedExpansion 命令打開延遲擴展可以讓批次檔中的變數，在執行時擴展變成值，而不是在命令被解析的時後。

使用 FOR 迴圈或是 if 括號表達式等循環命令，延遲擴展將允許您始終讀取變數的當前值。

現在我們在程式的最上面加上 SETLOCAL EnableDelayedExpansion

> notepad test.bat

檔案內容

SETLOCAL EnableDelayedExpansion
set VAR=before
if "%VAR%" == "before" (
set VAR=after
if "%VAR%" == "after" @echo "If you see this, it worked"
)

執行程式

> test.bat

螢幕輸出

>test.bat
>SETLOCAL EnableDelayedExpansion
>set VAR=before
>if "before" == "before" (
set VAR=after
if "before" == "after"
)

奇怪 ??? 為甚麼還是沒有在螢幕上輸出 “If you see this, it worked” 這段字串 ?

答案是因為 : 要讓延遲擴展生效，必須使用!variable_name!，在變數的前後都加驚嘆號，來立即讀取變數的值。

修改程式

> notepad test.bat

編輯程式內容，將第二個 %VAR% 改成 !VAR!

@echo off
SETLOCAL EnableDelayedExpansion
set VAR=before
if "%VAR%" == "before" (
set VAR=after
if "!VAR!" == "after" @echo "If you see this, it worked"
)

執行程式

> test.bat

螢幕輸出

"If you see this, it worked"

成功 ~

參考文章

技術重點 2 - Windows 遠端啟動 VMware 虛擬電腦

1. 使用 psexec 必須使用此設定

開放遠端連線權限，用系統管理員身份打開 CMD，並執行以下命令

reg add HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\system /v LocalAccountTokenFilterPolicy /t REG_DWORD /d 1 /f

[註] 1. 機碼修改後，需重啟電腦 2. 如有安裝 F-Secure，請關閉 F-Secure 防火牆參考文章連結 : PsTools 於 Windows 10 的環境設定

2. 執行 `psexec` 簡易命令

打開 CMD ，切換到 CNT.2023.v4.5\D9-1A2M3W工作目錄，執行以下測試命令

psexec \\192.168.39.55 -s "hostname"

螢幕輸出

PsExec v2.4 - Execute processes remotely
Copyright (C) 2001-2022 Mark Russinovich
Sysinternals - www.sysinternals.com
DESKTOP-VNDI6BN
hostname exited on 192.168.39.55 with error code 0.

3. 啟動虛擬機器

$ psexec \\120.96.143.3 -s -d "C:\Program Files (x86)\VMware\VMware Player\vmrun" "start" "C:\Users\bigred\alp.k1a1\alpine64.vmwarevm\alpine64.vmx" "nogui"

參數解釋 :

-s，遠端執行的命令將會由 System account 執行
-d，我們連線的實體主機即使斷線，所執行的命令不會因此中斷。您無法獲得任何退出代碼或輸出，因為遠程 PsExec 服務只是啟動程序，將新程序 ID 告知原始 PsExec 程序，而不再關心新程序的作用。

[stackoverflow 參考文章] What does psexec -s do?

4. 如何知道執行 `psexec` 成功

打開工作管理員，確認有沒有以下的顯示

5. 關閉虛擬機器

方法一 : 直接下命令

psexec \\120.96.143.5 -s -d "C:\Program Files (x86)\VMware\VMware Player\vmrun" "stop" "C:\Users\bigred\alp.k1w3\alpine64.vmwarevm\alpine64.vmx"

方法二 : 使用工作管理員在 vmware-vmx.exe 按右鍵結束工作

將 *.lck檔和類型是 VMEM檔案 的三個檔案刪掉

psexec 參考連結

Everything You Wanted to know About Psexec

技術參考連結

2023 Windows & VMware IaC 開發及使用手冊

Wed, 22 Feb 2023 16:02:50 +0800

認識 IaC

伺服器環境的手動配置給 IT 專業人員帶來了一些挑戰，例如，由於不斷增加的複雜性和對 IT 基礎架構的相關需求。

Infrastructure as Code，中文翻為 “基礎架構程式碼” ，提供了一種替代方法，將基礎架構的提供、配置與管理方式自動化。

Iac 分為兩種方法，分別是 Declarative (聲明式) 和 Imperative (宣告式)

聲明式主要說明基礎系統架構要完成甚麼目標、需要的資源或任何狀態
宣告式定義了實現基礎系統架構所需的特定命令，並以正確的順序執行這些命令。

執行它的好處不僅可以降低 Human Error 、縮短建置系統的時間，還可以確保每次都建立出一模一樣的基礎架構。

舉例，在 CNT.2023.v4.5 的套件包裡面的 docker.bat, k8s.bat, dt.bat …等批次檔案，都能快速建立起，對應的基礎架構。

註解：IaC 技術幫我們從落地雲主機自動產生以及複製虛擬主機

參考文章連結 1 : 何謂基礎架構程式碼 (IaC)？
參考文章連結 2 : What is Infrastructure as Code (IaC)?
參考文章連結 3 : Understanding Infrastructure as Code (IaC) in less than 10 minutes

Windows & VMware IaC 平台設計

平台目標

建置分散型 K8S 叢集系統架構，有 1M 2W，或是 3M 3W

重要操作步驟

K8S 叢集系統準備
設計叢集架構 (mactohost)
產生虛擬電腦資料夾
壓縮虛擬電腦資料夾
派送虛擬電腦資料夾
解壓縮所有虛擬主機壓縮檔
啟動與檢視所有虛擬主機
關閉所有虛擬主機
備份與還原所有虛擬主機
清除所有虛擬主機

實體電腦硬體規格

多台實體 Windows 電腦
記憶體至少 24 G
CPU: 4 C / 2 HT
須為靜態 IPv4 的 IP 位址
每台實體 Windows 10 電腦會分別跑兩台不同的虛擬電腦

虛擬電腦硬體規格

Alpine Linux 3.17.1 : 6 台
記憶體 : 實體 Windows 電腦總記憶體(GB) / 3
CPU : 2 vCPU
網路模式 : bridge

虛擬電腦網路設定

根據叢集的大小來規劃 IP 位址、 Subnetmask 以及 Default Gateway

開始建置 k1 Cluster (1A1M4W)

1. K8S 叢集系統準備

確認實體 Windows 10 主機之間可用 SSH 連接 測試命令 :

> ssh student@<附近好同學的實體 Windows 10 電腦 的 IP 位址>

如果無法連線，請用系統管理員執行 powershell，並執行以下命令 執行以下命令，檢查 OpenSSH 目前的狀態
```
Get-WindowsCapability -Online | Where-Object Name -like 'OpenSSH*'
```
如果 OpenSSH Server 沒有安裝，命令應返回以下輸出 :
```
Name : OpenSSH.Client~~~~0.0.1.0
State : Installed
Name : OpenSSH.Server~~~~0.0.1.0
State : NotPresent
```

安裝 OpenSSH Server 命令 :

Add-WindowsCapability -Online -Name OpenSSH.Server~~~~0.0.1.0

螢幕輸出 :

Path :
Online : True
RestartNeeded : False

啟動 sshd 服務命令

# Start the sshd service
Start-Service sshd
# OPTIONAL but recommended:
Set-Service -Name sshd -StartupType 'Automatic'
# Confirm the Firewall rule is configured. It should be created automatically by setup. Run the following to verify
if (!(Get-NetFirewallRule -Name "OpenSSH-Server-In-TCP" -ErrorAction SilentlyContinue | Select-Object Name, Enabled)) {
Write-Output "Firewall Rule 'OpenSSH-Server-In-TCP' does not exist, creating it..."
New-NetFirewallRule -Name 'OpenSSH-Server-In-TCP' -DisplayName 'OpenSSH Server (sshd)' -Enabled True -Direction Inbound -Protocol TCP -Action Allow -LocalPort 22
} else {
Write-Output "Firewall rule 'OpenSSH-Server-In-TCP' has been created and exists."
}

連線附近好同學的實體 Windows 10 電腦的 IP 位址測試

> ssh student@<附近好同學的實體 Windows 10 電腦 的 IP 位址>

Install OpenSSH Server on Windows 10 官網連結

將連接 Windows 的帳號密碼存入 passwd 檔案

> notepad %USERPROFILE%\CNT.2023.v4.5\K1-1A1M4W\passwd

將連接 Alpine Linux 的帳號密碼存入 passwd_alp 檔案
```
> notepad %USERPROFILE%\CNT.2023.v4.5\K1-1A1M4W\passwd_alp
```
VMware Player 這軟體一定要做以下設定, 否則建立虛擬主機的目錄會殘溜目錄 4-1. Player > File > Preferences... 4-2. In the “Software updates” group > uncheck the “Check for software components as needed” option.
Windows 主機需安裝 7z
確認被遠端控制的實體 Windows 主機，開放遠端登入權限 (需用管理員身分打開 CMD 執行以下命令)
```
reg add HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\system /v LocalAccountTokenFilterPolicy /t REG_DWORD /d 1 /f
```
注意 ! 機碼修改後，需重啟電腦 !

2. 設計 k1 叢集架構

設定 k1 叢集架構檔 - `mactohost`

在 Windows 中，打開 CMD ，執行以下命令

切換至工作目錄

> cd CNT.2023.v4.5\K1-1A1M4W

編輯 mactohost

> notepad k1mactohost

檔案內容 :

# Default Cluster
00:00 x1 none
00:01 x2 VMware
00:02 x3 VMware
00:03 a1 admin
00:04 m1 master
00:05 m2 master
00:06 m3 master
00:07 w1 worker
00:08 w2 worker
00:09 w3 worker
# K8S k1 Cluster
10:d2 k1a1 admin k1a1.210.zip k1/alp.k1a1 192.168.39.55
10:d3 k1m1 master k1m1.211.zip k1/alp.k1m1 192.168.39.55
10:d4 k1m2 master
10:d5 k1m3 master
10:d6 k1e1 etcd
10:d7 k1w1 worker k1w1.215.zip k1/alp.k1w1 192.168.39.55
10:d8 k1w2 worker k1w2.216.zip k1/alp.k1w2 192.168.39.55
10:d9 k1w3 worker k1w3.217.zip k1/alp.k1w3 192.168.39.55
10:da k1w4 worker k1w4.218.zip k1/alp.k1w4 192.168.39.55
10:db k1w5 worker

欄位說明 :

第一欄為設定 VM 主機的 Mac Address 最後兩個位元 (byte)
- 用冒號分隔的格式 : 前半部是 k1 = 10, k2 = 20… 依此類推後半部是 VM 的 IP 位址尾數換算為 16 進位
- 舉例 : 假設現在要設定的是 k1 叢集裡面 IP 位址尾數 .210 的 VM，依照上述規則，得到的答案 : 10:d2
第二欄為設定 VM 主機名稱
- 格式 : 叢集名稱 + mactohost 第三欄位的第一個字 + 序號(從 1 開始)
第三欄為各 VM 的角色扮演
第四欄為 VM 目錄壓縮檔名
- 格式 : VM 主機名稱 + VM IP 位址的尾數 + 壓縮檔的附檔名 (.zip)
第五欄為 VM 存放的目錄名稱
- 格式 : 叢集名稱/alp.主機名稱 (alp = Alpine Linux的縮寫)
第六欄為 VM 要分配到哪一台實體電腦的 IP 位址

3. 建立 k1 叢集所有虛擬主機

k1create 程式概述：在落地雲主機的 C:\Users\bigred\CNT.2023.v4.5\K3-1A1M4W 目錄下，做出 k1 的資料夾，並在 k1 目錄下建立 6 台虛擬機

@echo off
REM 設定 Cluster Name
for /f "tokens=1" %%y in ('dir %CD% /B ^| findstr "mactohost"') do (
set CLUSTERNAME=%%y
)
for /f "tokens=3 delims= " %%z in ('type %CLUSTERNAME:~0,2%mactohost ^| more +11 ^| findstr "#"') do (
set CN=%%z
)
Rem/||(
判斷 Cluster Name 有沒有設定錯誤
- %Variable:~0,2%
是取變數值的前兩個字
- GOTO:eof
在不定義標籤的情況下將控制傳送到當前批次檔的末端，此時，批次檔會自行退出。
)
if NOT %CLUSTERNAME:~0,2% == %CN% (
echo "Cluster Name Error !"
GOTO:eof
)
REM 判斷在當前目錄下是否存在 %CN% (有可能是 k1 或 k2 或是 k*)目錄
if exist %CN%\ (
echo "%CN% Folder existed !"
GOTO:eof
)
REM 設定 VM 可用記憶體範圍
powershell -Command "(Get-CimInstance Win32_PhysicalMemory | Measure-Object -Property capacity -Sum).sum /3Mb" > max.txt
powershell -Command "(Get-CimInstance Win32_PhysicalMemory | Measure-Object -Property capacity -Sum).sum /8Mb" > min.txt
set /P MAX=<max.txt
set /P MIN=<min.txt
set RSMMAX='memsize = \"4096\"', 'memsize = \"%MAX%\"'
set RSMMIN='memsize = \"4096\"', 'memsize = \"%MIN%\"'
REM 修改 VM 的 MAC Address 必要設定
set RSADR1='ethernet0.generatedAddressOffset = \"0\"'
set RSADR2='ethernet0.addressType = \"generated\"' , 'ethernet0.addressType = \"static\"'
REM 將 VM 網路模式設為 bridge 橋接模式
set RSTYPE='ethernet0.connectionType = \"nat\"' , 'ethernet0.connectionType = \"bridge\"'
REM 啟動 ALP.K8S 虛擬主機
"C:\Program Files (x86)\VMware\VMware Player\vmrun" start "%USERPROFILE%\CNT.2023.v4.5\ALP.K8S\alpine64.vmwarevm\alpine64.vmx" > nul
timeout /t 60 > nul
Rem/||(
抓 ALP.K8S 虛擬主機的 IP 位址，將命令執行結果重導到 admip.txt 這個檔案中
- 透過 set /P 將 admip.txt 檔案中第一行的內容，設為 admip 變數的值
- 檔案中的第二行以下，包含第二行的內容，都會被丟棄
)
"C:\Program Files (x86)\VMware\VMware Player\vmrun" getGuestIPAddress "%USERPROFILE%\CNT.2023.v4.5\ALP.K8S\alpine64.vmwarevm\alpine64.vmx" > admip.txt
set /P admip=<admip.txt
Rem/||(
上載 mactohosts 至 ALP.K8S 虛擬主機
- 變數 ERRORLEVEL 為上一行命令執行完後回傳的值 ^(exit code^)，0 代表執行成功。
- EQU ^(Equal To^)執行數字比較，如果需要字符串比較，使用 == 比較運算符。
- USERPROFILE 變數為顯示當前使用者的家目錄，例如 : C:\USERS\bigred
)
echo y | pscp -pwfile "passwd_alp" ".\%CN%mactohost" bigred@%admip%:/home/bigred/bin/mactohost 2> nul > nul
if %ERRORLEVEL% EQU 0 echo "%CN%mactohost scp ok"
echo y | pscp -pwfile "passwd_alp" bigred@%admip%:/home/bigred/.ssh/* %USERPROFILE%\.ssh\ 2> nul > nul
if %ERRORLEVEL% EQU 0 echo "%admip% .ssh/* scp ok"
del %USERPROFILE%\.ssh\known_hosts 2> nul
del %USERPROFILE%\.ssh\known_hosts.old 2> nul
ssh -o "ConnectTimeout=5" -o "StrictHostKeyChecking=no" bigred@%admip% "sudo apk update; sudo apk upgrade" 2> nul > nul
if %ERRORLEVEL% EQU 0 echo "System upgrade ok"
"C:\Program Files (x86)\VMware\VMware Player\vmrun" stop "%USERPROFILE%\CNT.2023.v4.5\ALP.K8S\alpine64.vmwarevm\alpine64.vmx" > nul
timeout /t 60 > nul
Rem/||(
建立所有 K8S Cluster 的 VM
- SETLOCAL ENABLEDELAYEDEXPANSION 為 啟用變數延遲展開
- 意思是 變數會在執行時才會展開變成它的值，而不是在解析時就變成值
- 要延遲展開的變數格式由 %var% 更改為 !var!
- gc 為 powershell 的命令，用來顯示檔案內容，-replace 參數主要是替換一個文字檔中的字串
- Out-File 將輸出傳到一個檔案， -Encoding 參數將輸出轉換為指定格式
)
setlocal EnableDelayedExpansion
if exist %CN%mactohost (
if not exist %CN%\ (
mkdir %CN% > nul
if %ERRORLEVEL% EQU 0 (
for /f "tokens=1-6 delims= " %%a in ('findstr 120 %CN%mactohost') do (
set x=%%~nxe
set RS='displayName = \"ALP.K8S\"', 'displayName = \"!x!\"'
set y=%%a
set RSMAC='ethernet0.generatedAddress = \"00:0c:29:.*\"' , 'ethernet0.address = \"00:50:56:ab:!y!\"'
mkdir %CN%\!x! > nul
xcopy /E /Y %USERPROFILE%\CNT.2023.v4.5\ALP.K8S %CN%\!x!\ > nul
powershell -Command "(gc %CN%\!x!\alpine64.vmwarevm\alpine64.vmx) -replace "!RS!" -replace "!RSMAC!" -replace "%RSMMAX%" -replace "%RSTYPE%" -replace "%RSADR1%" -replace "%RSADR2%" | Out-File -Encoding "UTF8" vm.temp"
del /Q %CN%\!x!\alpine64.vmwarevm\alpine64.vmx
copy /Y vm.temp %CN%\!x!\alpine64.vmwarevm\alpine64.vmx > nul
if !ERRORLEVEL! EQU 0 echo "%CN%\!x!\ (%MAX%) ok"
)
del *.txt > nul
del vm.temp > nul
)
)
)

執行建立 K1 叢集命令

在 Windows 工作目錄 CNT.2023.v4.5\K1-1A1M4W 下，執行程式

> create.bat

螢幕輸出

"k1mactohost scp ok"
"192.168.179.129 .ssh/* scp ok"
"System upgrade ok"
"k1\alp.k1a1\ (8192) ok"
"k1\alp.k1m1\ (8192) ok"
"k1\alp.k1w1\ (8192) ok"
"k1\alp.k1w2\ (8192) ok"
"k1\alp.k1w3\ (8192) ok"
"k1\alp.k1w4\ (8192) ok"

[重要] 根據實體電腦的記憶體來修改 k1create.bat 中 RSMMAX 變數直的大小 [註] 如遇到需要輸入密碼，請參考以下連結 : Windows SSH: Permissions for ‘private-key’ are too open

4. 壓縮 k1 叢集所有虛擬主機

@echo off
REM 設定 Cluster Name
for /f "tokens=1" %%y in ('dir /B ^| findstr "mactohost"') do (
set CN=%%y
)
REM 在 mactohost 中抓取壓縮時需要的字串
if exist %CN:~0,2%zip.txt (del %CN:~0,2%zip.txt > nul)
for /f "tokens=3,* delims= " %%a in (%CN:~0,2%mactohost) do echo %%b | findstr zip >> %CN:~0,2%zip.txt
REM 透過 For 迴圈，逐行代變數，壓縮 VM 所在的資料夾
setlocal EnableDelayedExpansion
for /f "tokens=1,2 delims= " %%a in (%CN:~0,2%zip.txt) do (
7z\7z -mx1 a %CN:~0,2%\%%a %%b > nul
if !ERRORLEVEL! EQU 0 (echo "%%a ok")
)
del %CN:~0,2%zip.txt > nul

執行壓縮程式

> zip.bat

螢幕輸出

"k1a1.50.zip ok"
"k1m1.51.zip ok"
"k1w1.55.zip ok"
"k1w2.56.zip ok"
"k1w3.57.zip ok"
"k1w4.58.zip ok"

5. 派送 k1 叢集所有虛擬主機

@echo off
for /f "tokens=1" %%y in ('dir /B ^| findstr "mactohost"') do (
set CN=%%y
)
if exist %CN:~0,2%scp.txt (del %CN:~0,2%scp.txt > nul)
for /f "tokens=1* delims= " %%a in (%CN:~0,2%mactohost) do echo %%b | findstr zip >> %CN:~0,2%scp.txt
setlocal EnableDelayedExpansion
for /f "tokens=3,5 delims= " %%a in (%CN:~0,2%scp.txt) do (
echo y | pscp -scp -pwfile passwd "%CN:~0,2%\%%a" bigred@%%b:"%%a" 2> nul > nul
if !ERRORLEVEL! EQU 0 (
echo "%%a to %%b ok"
) else (
echo "%%a to %%b failed"
)
)
del %CN:~0,2%scp.txt > nul

執行程式

> scp.bat

螢幕輸出

"k1a1.50.zip to 120.96.143.3 ok"
"k1m1.51.zip to 120.96.143.3 ok"
"k1w1.55.zip to 120.96.143.4 ok"
"k1w2.56.zip to 120.96.143.4 ok"
"k1w3.57.zip to 120.96.143.5 ok"
"k1w4.58.zip to 120.96.143.5 ok"

6. 解壓縮所有虛擬主機壓縮檔至落地雲主機

@echo off
for /f "tokens=1" %%y in ('dir /B ^| findstr "mactohost"') do (
set CN=%%y
)
for /f "skip=12 tokens=4-6 delims= " %%a in (%CN:~0,2%mactohost) do (
for /f "tokens=1 delims= " %%i in ('PsExec.exe -accepteula -nobanner \\%%c -u bigred -p "!QAZ2wsx " -s "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" "-Command" "Test-Path" "-Path" "C:\Users\bigred\%CN:~0,2%\%%~nxb" ^2^>nul') do (
if %%i == True (
psexec -accepteula -nobanner \\%%c -u bigred -p "!QAZ2wsx " -s "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" "-Command" "Remove-Item" "C:\Users\bigred\%CN:~0,2%\%%~nxb" "-Recurse" 2> nul
psexec -accepteula -nobanner \\%%c -u bigred -p "!QAZ2wsx " -s "powershell" "-Command" "Expand-Archive" "-LiteralPath" "C:\Users\bigred\%%a" "-DestinationPath" "C:\Users\bigred" 2> nul
) else (
psexec -accepteula -nobanner \\%%c -u bigred -p "!QAZ2wsx " -s "powershell" "-Command" "Expand-Archive" "-LiteralPath" "C:\Users\bigred\%%a" "-DestinationPath" "C:\Users\bigred" 2> nul
)
)
for /f "tokens=1 delims= " %%i in ('PsExec.exe -accepteula -nobanner \\%%c -u bigred -p "!QAZ2wsx " -s "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" "-Command" "Test-Path" "-Path" "C:\Users\bigred\%CN:~0,2%\%%~nxb" ^2^>nul') do (
if %%i == True (
echo "%%c %CN:~0,2%\%%~nxb unzip ok !"
) else (
echo "%%c %CN:~0,2%\%%~nxb unzip Failed !"
)
)
)

[註] %%~nxb，在範例中 %%b 為包含目錄的檔案名稱 : k1/alp.k1w1， %%~nb - 顯示 alp (%%b 的檔名) %%~xb - 顯示 .k1w1 ( %%b 的副檔名，包含 .)

執行解壓縮程式

> unzip.bat

螢幕輸出

"120.96.143.3 k1\alp.k1a1 unzip ok !"
"120.96.143.3 k1\alp.k1m1 unzip ok !"
"120.96.143.4 k1\alp.k1w1 unzip ok !"
"120.96.143.4 k1\alp.k1w2 unzip ok !"
"120.96.143.5 k1\alp.k1w3 unzip ok !"
"120.96.143.5 k1\alp.k1w4 unzip ok !"

7. 啟動與檢視 k1 叢集所有虛擬主機

@echo off
for /f "tokens=1" %%y in ('dir /B ^| findstr "mactohost"') do (
set CN=%%y
)
setlocal EnableDelayedExpansion
for /f "skip=12 tokens=4-6 delims= " %%a in (%CN:~0,2%mactohost) do (
psexec \\%%c -accepteula -nobanner -u bigred -p "!QAZ2wsx " -s "C:\Program Files (x86)\VMware\VMware Player\vmrun" "start" "C:\Users\bigred\%CN:~0,2%\%%~nxb\alpine64.vmwarevm\alpine64.vmx" "nogui" 2> nul > nul
if !ERRORLEVEL! EQU 0 (
echo "%CN:~0,2%\%%~nxb started"
) else (
echo "%CN:~0,2%\%%~nxb failed"
)
timeout /t 5 > nul
)

1. 執行啟動 k1 叢集所有虛擬主機命令

> start.bat

螢幕輸出

"k1\alp.k1a1 started"
"k1\alp.k1m1 started"
"k1\alp.k1w1 started"
"k1\alp.k1w2 started"
"k1\alp.k1w3 started"
"k1\alp.k1w4 started"

2. 在 Windows 檢測已開啟的虛擬電腦

@echo off
for /f "tokens=1" %%y in ('dir /B ^| findstr "mactohost"') do (
set CN=%%y
)
if exist %CN:~0,2%listp.txt (del %CN:~0,2%list.txt > nul)
for /f "tokens=1* delims= " %%a in (%CN:~0,2%mactohost) do echo %%a %%b | findstr zip >> %CN:~0,2%list.txt
for /f "tokens=1,2 delims= " %%a in (%CN:~0,2%list.txt) do (
for /f "tokens=1,2 delims=:" %%i in ("%%a") do (
ping -n 2 "120.96.143.0x%%j" | find "TTL" 2> nul > nul
if errorlevel 1 (
echo "%%b not running"
) else (
ssh -o "ConnectTimeout=5" -o "StrictHostKeyChecking=no" bigred@"120.96.143.0x%%j" "n=$(hostname;hostname -i); echo $n running" 2> nul
)
)
)
del *.txt > nul

執行檢測命令

> list.bat

螢幕輸出 :

k1a1 120.96.143.50 running
k1m1 120.96.143.51 running
k1w1 120.96.143.55 running
k1w2 120.96.143.56 running
k1w3 120.96.143.57 running
k1w4 120.96.143.58 running

8. 關閉 k1 叢集所有虛擬主機

@echo off
for /f "tokens=1" %%y in ('dir /B ^| findstr "mactohost"') do (
set CN=%%y
)
if exist %CN:~0,2%stop.txt (del %CN:~0,2%stop.txt > nul)
for /f "tokens=1* delims= " %%a in (%CN:~0,2%mactohost) do echo %%a %%b | findstr zip >> %CN:~0,2%stop.txt
for /f "delims= " %%a in (%CN:~0,2%stop.txt) do (
for /f "tokens=1,2 delims=:" %%i in ("%%a") do (
ping -n 2 "120.96.143.0x%%j" | find "TTL" 2> nul > nul
if errorlevel 1 (
echo "120.96.143.0x%%j not running"
) else (
ssh -o "ConnectTimeout=5" -o "StrictHostKeyChecking=no" bigred@"120.96.143.0x%%j" "sudo poweroff; exit" 2> nul > nul
if ERRORLEVEL 0 echo "120.96.143.0x%%j stoped"
)
)
)
del *.txt > nul

執行關機 k1 叢集所有虛擬主機命令

> stop.bat

螢幕輸出

"120.96.143.0x32 stoped"
"120.96.143.0x33 stoped"
"120.96.143.0x37 stoped"
"120.96.143.0x38 stoped"
"120.96.143.0x39 stoped"
"120.96.143.0x3a stoped"

9. 備份與還原 k1 叢集所有虛擬主機

@echo off
for /f "tokens=1" %%y in ('dir /B ^| findstr "mactohost"') do (
set CN=%%y
)
SET /P an="Would you want to stop all %CN:~0,2% Cluster VM (do/done)"
if %an% == do (
CALL %CN:~0,2%stop.bat
CALL :backup
GOTO:eof
) else (
CALL :backup
GOTO:eof
)
:backup
setlocal EnableDelayedExpansion
for /f "skip=12 tokens=4-6 delims= " %%a in (%CN:~0,2%mactohost) do (
for /f "tokens=1 delims= " %%i in ('PsExec.exe -accepteula -nobanner \\%%c -s "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" "-Command" "Test-Path" "-Path" "C:\Users\bigred\%CN:~0,2%\%%~nxb" ^2^>nul') do (
if %%i == True (
psexec -accepteula -nobanner \\%%c -s "%ProgramFiles%\7-Zip\7z" "-mx1" "a" "C:\Users\bigred\%%a.bak" "C:\Users\bigred\%%b" 2> nul > nul
if !ERRORLEVEL! EQU 0 (
echo "%%a.bak backup ok !"
) else (
echo "%%a.bak backup failed !"
)
) else (
echo "VM folder not found !"
GOTO:eof
)
)
)

執行備份 k1 叢集所有虛擬主機命令

> backup.bat

螢幕輸出 :

 Would you want to stop all k1 Cluster VM (do/done)

回答 do –> 會先將 k1 叢集的所有虛擬主機關機後再備份

螢幕輸出 :

"120.96.143.0x32 stoped"
"120.96.143.0x33 stoped"
"120.96.143.0x37 stoped"
"120.96.143.0x38 stoped"
"120.96.143.0x39 stoped"
"120.96.143.0x3a stoped"
"k1a1.50.zip.bak backup ok !"
"k1m1.51.zip.bak backup ok !"
"k1w1.55.zip.bak backup ok !"
"k1w2.56.zip.bak backup ok !"
"k1w3.57.zip.bak backup ok !"
"k1w4.58.zip.bak backup ok !"

回答 done –> 會直接開始備份 k1 叢集所有虛擬主機

螢幕輸出 :

"k1a1.50.zip.bak backup ok !"
"k1m1.51.zip.bak backup ok !"
"k1w1.55.zip.bak backup ok !"
"k1w2.56.zip.bak backup ok !"
"k1w3.57.zip.bak backup ok !"
"k1w4.58.zip.bak backup ok !"

[註] 新的備份檔會直接覆蓋掉舊的備份檔

執行還原 k1 叢集所有虛擬主機命令

> restore.bat

執行命令後，會先問要不要先關機，要就回答 do 不要就回答 done

螢幕輸出 :

Would you want to stop all k1 Cluster VM (do/done)done
"120.96.143.3 k1\alp.k1a1 restore ok !"
"120.96.143.3 k1\alp.k1m1 restore ok !"
"120.96.143.4 k1\alp.k1w1 restore ok !"
"120.96.143.4 k1\alp.k1w2 restore ok !"
"120.96.143.5 k1\alp.k1w3 restore ok !"
"120.96.143.5 k1\alp.k1w4 restore ok !"

10. 清除 k1 叢集所有虛擬主機

@echo off
for /f "tokens=1" %%y in ('dir /B ^| findstr "mactohost"') do (
set CN=%%y
)
REM del all include k1* exclude *.bak
SET /P an="Would you want to stop all %CN:~0,2% Cluster VM (do/done)"
if %an% == do (
CALL %CN:~0,2%stop.bat
CALL :del_k8s
GOTO:eof
) else (
CALL :del_k8s
GOTO:eof
)
:del_k8s
if exist ip.txt (del ip.txt)
for /f "skip=12 tokens=6 delims= " %%y in (%CN:~0,2%mactohost) do echo %%y >> ip.txt
setlocal EnableDelayedExpansion
for /f "tokens=1 delims= " %%x in ('type ip.txt ^| sort.exe /unique') do (
psexec -accepteula -nobanner \\%%x -s "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" "-Command" "Remove-Item" "C:\Users\bigred\%CN:~0,2%" "-Recurse" 2> nul
if !ERRORLEVEL! EQU 0 (
echo "%%x %CN:~0,2% folder deleted !"
) else (
echo "%%x %CN:~0,2% folder does not exist !"
)
)
for /f "skip=12 tokens=4-6 delims= " %%a in (%CN:~0,2%mactohost) do (
psexec -accepteula -nobanner \\%%c -s "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" "-Command" "Remove-Item" "C:\Users\bigred\%%a" 2> nul
if !ERRORLEVEL! EQU 0 (
echo "%%c %%a deleted !"
) else (
echo "%%c %%a does not exist !"
)
)
del ip.txt 2> nul > nul
RD /S /Q /S /Q %CN:~0,2% 2> nul
if %ERRORLEVEL% == 0 echo "local %CN:~0,2% folder deleted !"

執行清除 k1 叢集所有虛擬主機命令

> del.bat

執行命令後，會先問要不要先關機，要就回答 do 不要就回答 done

螢幕輸出 :

Would you want to stop all k1 Cluster VM (do/done)done
"120.96.143.3 k1 folder deleted !"
"120.96.143.4 k1 folder deleted !"
"120.96.143.5 k1 folder deleted !"
"120.96.143.3 k1a1.50.zip deleted !"
"120.96.143.3 k1m1.51.zip deleted !"
"120.96.143.4 k1w1.55.zip deleted !"
"120.96.143.4 k1w2.56.zip deleted !"
"120.96.143.5 k1w3.57.zip deleted !"
"120.96.143.5 k1w4.58.zip deleted !"
"local k1 folder deleted !"

[註] 會刪除 k1 叢集所有虛擬主機、壓縮檔以及本地的 k1 資料夾，但不包含備份的 (*.bak) 檔

IaC 平臺開發關鍵技術

技術重點 1 - Windows batch script - 透過 `FOR` 命令可以將文字檔中的欄位資料取出

:star: 首先要注意 ! 如果要在批次程式中使用 FOR 命令，請指定 %%variable，而不要指定 %variable。變數名稱有大小寫的區分，所以 %i 不同於 %I。

逐行讀取文字檔的 `for /F` 迴圈

/f ，可以對 for 迴圈傳入的檔案、字串或命令，進行解析，以下為讀取檔案的範例 :

for /f "tokens=3,* delims= " %%a in (k1mactohost) do echo %%b

概述：逐行讀取 k1mactohost 檔案的內容，然後依照分隔符號 (空格) 將檔案內容的值分別設為指定的變數，再將指定的變數內容輸出在螢幕上。
tokens=x,y,m-n 用來決定一次要取出幾個欄位，第一個欄位會存放在第一個自動變數，第二個欄位會存放在第二個自動變數裡，依此類推。在範例中為tokens=3,*，代表將第三個欄位的值存放在變數 a ，其他剩餘欄位的值會放在變數 b 註: x, x, m, n 都為數字，只是 m-n 是一種特別的表示法，代表一個解析欄位的範圍。例如 tokens=2-4 就代表取得第2、第 3 與第 4 個欄位的值並分別塞指定的變數裡
delims=xxx 用來決定欄位的分隔符號，預設為空格與 TAB 符號，並可自訂多個符號。在範例中，將欄位的分隔符號設為空格註: xxx 代表多個字元，如果要同時設定小數點 . 與逗號 , 就要輸入delims=.,

練習

有一個文字檔 tip.txt

內容是

10:6e d8a5 120.96.143.22
10:6f d8a6 120.96.143.22

寫一隻程式（tip.bat）用冒號分割顯示以下的結果：

6e
6f

答案： $ notepad tip.bat

@echo off
for /f "tokens=1 delims= " %%a in (tip.txt) do echo %%a >> t1.txt
for /f "tokens=2 delims=:" %%b in (t1.txt) do echo %%b

$ tip.bat

6e
6f

參考文章

如何利用批次檔(Batch)讀取指令執行的結果或文字檔案內容 for 迴圈

技術重點 2 - Windows 遠端啟動 VMware 虛擬電腦

1. 使用 psexec 必須使用此設定

開放遠端登入權限

執行 regedit
找到 HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\policies\system 新增一個「DWORD 值」，並把它命名為「LocalAccountTokenFilterPolicy」，然後再把他的值設定成「1」。參考網址 : PsTools 於 Windows 10 的環境設定

2. 下載 `psexec.exe` 執行檔

下載 psexec v2.4 連結

$ psexec /accepteula

3. 執行 `psexec` 命令

啟動虛擬機器

$ psexec \\120.96.143.3 -u bigred -p "!QAZ2wsx " -s -d "C:\Program Files (x86)\VMware\VMware Player\vmrun" "start" "C:\Users\bigred\alp.k1a1\alpine64.vmwarevm\alpine64.vmx" "nogui"

-s system account -d 我們連線的實體主機即使斷線，所執行的命令不會因此中斷

4. 如何知道執行 `psexec` 成功

打開工作管理員，確認有沒有以下的顯示

5. 關閉虛擬機器

直接下命令

psexec \\120.96.143.5 -u bigred -p "!QAZ2wsx " -s -d "C:\Program Files (x86)\VMware\VMware Player\vmrun" "stop" "C:\Users\bigred\alp.k1w3\alpine64.vmwarevm\alpine64.vmx"

使用工作管理員在 vmware-vmx.exe 按右鍵結束工作
把三個檔案刪掉

參考連結

:book: Everything You Wanted to know About Psexec

技術參考連結

[註]   代表顯示一個空白符號 https://www.w3schools.com/html/html_entities.asp

VMware 壓縮虛擬硬碟 with Ubuntu

Sun, 07 Aug 2022 14:28:36 +0800

# 更新系統套件
$ sudo apt update; sudo apt -y upgrade; sudo apt -y autoremove
# 安裝 open-vm-tools
$ sudo apt install open-vm-tools
# 重開 VM
$ sudo reboot
# 將 windows 的 .vmdk 檔壓縮
$ sudo /usr/bin/vmware-toolbox-cmd disk shrink /

注意！要關 3 D 圖形！

tags: `VMware`

Linux KVM

Thu, 05 May 2022 17:33:16 +0800

介紹KVM

KVM是linux的核心技術，它可以啟動虛擬機器，和虛擬化技術，市面上所有linux都有這個功能，linux kennel是由Linus Torvalds寫的，他在2006年12月將Linux KVM寫進linux系統，只要是linux系統，CPU是intel的一定要有VT-X的功能，AMD的話要有開啟SVM功能，就能啟動KVM

我們透過VMware player建立Ubuntu或是Alpine虛擬機器，只要在原生的Linux系統上有啟動核心KVM的功能，這時候Linux系統搖身一變，變成可以啟動很多很多虛擬機器的平台。

只要是linux系統，有linux kernel，CPU具備VT-x功能(or SVM)，那麼你就可以直接啟動KVM。

硬體輔助虛擬化（Hardware-assisted virtualization）

Intel Virtualization Technology（VT-x）
AMD Secure Virtual Machine（SVM）

有此技術，各家虛擬技術均可稱為完全虛擬化，它們的差別在週邊硬體（網卡、硬碟）虛擬化支援，這兩個技術會直接穿透OS，直接使用CPU和記憶體，但其他的硬體周邊不支援

用穿透是比較誇張的詞，還是需要透過Hypervisor來調節資源，只是能夠用到真實的硬體而不是模擬的。

過去的專有名稱半虛擬化 : 需修改系統核心（Windows 不支援）完全虛擬化 : 需執行 CPU 指令轉碼如果有看到以上兩個名詞跟硬體輔助虛擬化比較的文章再看

QEMU

早期硬體沒有硬體輔助虛擬，所以使用 qemu 啟動虛擬機，後來硬體輔助虛擬出現，就可以讓 qemu 搭配 kvm

QEMU他是虛擬技術的一個軟體，這個軟體能夠幫你產生一台虛擬機器，但是他的CPU,記憶體,硬碟和網卡都是假的，此時搭配KVM產生的虛擬機器，代表底層有啟動VT-X或SVM，這台機器的CPU,記憶體就是真的，也就是用實體機器的

KVM運作架構

Hareware：指的是CPU,記憶體,網路和主機板…等
在作業系統啟動KVM的話，建立出來的VM，就可以穿透作業系統，直接使用CPU和記憶體，這樣的速度很快
除了CPU記憶體，用QEMU模擬出硬碟或網卡，這時候硬碟和網卡就是假的，就會比較慢
VirtIO是一個虛擬機器周邊的加速引擎，他可以幫我們用QEMU模擬出來的硬碟和網卡加速，但啟動前有一個先決條件，Intel的CPU一定要啟動VT-d這項技術，（本次課程練習不到）

GCP建VM

先連到GCP，在Compute Engine裡的設定 »> 把預設位置改成台灣，區域Ｃ

建立執行個體機器名稱: mykvm 機器設定，選運算最佳化系列：C2 機器類型：c2-standard-4（4個vCPU，16 GB記憶體）開機磁碟選作業系統：Ubuntu 版本：Ubuntu 20.04 LTS 開機磁碟類型：SSD 永久磁碟大小（GB）: 100 GB 好了後按選取

KVM-Check

先連線進去做一開始的環境設定

sudo nano /etc/ssh/sshd_config

sudo systemctl restart ssh

sudo passwd sreantony

sudo passwd root

sudo nano /etc/sudoers

在落地雲的CMD，ssh連進去 ``

$ sudo apt-get update
$ sudo apt-get install cpu-checker
........
0 upgraded, 2 newly installed, 0 to remove and 130 not upgraded.
Need to get 16.9 kB of archives.
After this operation, 66.6 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
$ kvm-ok
INFO: Your CPU does not support KVM extensions
INFO: For more detailed results, you should run this as root

Hypervisor

虛擬機器的建立,開啟,關閉,刪除,還有虛擬機器和虛擬機器之間的隔離，以上這些技術要完成的就是Hypervisor，虛擬機器的一生，要怎麼去管理和監控都是靠Hypervisor，如果沒有它，以上這些事情通通都不能做

分為兩個架構

Type 1 Hypervisor 裸機架構（Bare-Metal）企業會使用這種架構，作業系統會內建Hypervisor，舉個例字：Linux KVM，只要你是linux系統，有linux kernrl ，那一定會有KVM，這個架構就可以統稱裸機架構，有了這種架構，你就可以直接啟動不同的作業系統
Type 2 Hypervisor 寄居架構（Hosted）作業系統沒有內定Hypervisor，在作業系統上額外安裝Hypervisor就是寄居架構，這時候運作效率就會變慢，因為它還要透過作業系統才能做管理，像是我們在落地雲windows電腦裡面安裝VMware就是，這種寄居架構

巢狀虛擬化

會有很多個Hypervisor，在虛擬機器裡面再開一個虛擬機器，一般企業不允許，在GCP裡面可以突破這個限制，在GCP裡面建立一台虛擬機器，裡面再建立一台虛擬機器

GCP 巢狀虛擬化的限制

一定要是linux作業系統
機器類型不能使用E2和N2D

設定GCP巢狀虛擬

第一步啟動Cloud shell

用cloud shell的原因是因為不用額外安裝gcloud這個命令先點這個開啟編輯器把終端機的功能打開會呈現以下畫面

這就是gcp裡面的一台機器，我們已經透過cloud shell連進去這台機器了這台機器不用錢，我們用這台虛擬機器是因為google在管理虛擬機器，有一項命令叫做gcloud，這樣就不用額外安裝

第二部匯出

先下一個環境變數

$ export vmname='mykvm'

檢查區域

$ gcloud compute instances export ${vmname} \
--destination=${vmname}.yaml \ #設定等等匯出的名字叫mykvm.yaml
--zone=**asia-east1-c** #在gcloud上面要匯出你的VM的設定檔，一定會綁定機器在哪個地區

用tee命令在yaml檔後面再加兩筆資料，這就是巢狀虛擬化必須啟動的功能，可以用kvm-ok確定要不要設定

$ tee -a ${vmname}.yaml<<EOF
advancedMachineFeatures:
enableNestedVirtualization: true

把剛剛設定好的yaml檔，傳回去給gcp，並且請他重新開機立即生效

$ gcloud compute instances update-from-file ${vmname} \
--source=${vmname}.yaml \
--most-disruptive-allowed-action=RESTART \
--zone=asia-east1-c

好了後，再用ssh命令連進去機器，注意外部IP這時候有可能會更動

$ kvm-ok
INFO: /dev/kvm exists
KVM acceleration can be used

安裝Kvm

$ sudo apt install qemu-kvm
........
0 upgraded, 26 newly installed, 0 to remove and 130 not upgraded.
Need to get 19.6 MB of archives.
After this operation, 82.5 MB of additional disk space will be used.
Do you want to continue? \[Y/n\] y

$ sudo adduser ${USER} kvm
$ sudo reboot

檢查 kvm 是否正確安裝

$ kvm -version
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.21)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers

Alpine系統安裝

$ mkdir wk wk/iso wk/disk
$ cd wk
$ wget https://dl-cdn.alpinelinux.org/alpine/v3.15/releases/x86\_64/alpine-virt-3.15.4-x86\_64.iso -O iso/alpine-virt-3.15.4-x86_64.iso

使用qemu-img產生qcow2格式的硬碟

$ qemu-img create -f qcow2 disk/alpine.qcow2 10G
Formatting 'disk/alpine.qcow2', fmt=qcow2 size=10737418240 cluster\_size=65536 lazy\_refcounts=off refcount_bits=16

檢查硬碟格式

$ qemu-img info disk/alpine.qcow2
image: disk/alpine.qcow2
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 196 KiB

$ sudo kvm -m 2048 \
-cdrom iso/alpine-virt-3.15.4-x86_64.iso \
-hda disk/alpine.qcow2 \
-nographic
參數說明：
-m 虛擬機記憶體設定 (單位 MB)
-cdrom 虛擬機開機放入光碟片
-hda 虛擬機開機掛載硬碟 (代號 a)
-nographic 不啟動圖形化介面
備註：
開機順序 hd(a) -> cdrom

alpine設定

$ which kvm
/usr/bin/kvm

$ cat /usr/bin/kvm
#!/bin/sh
exec qemu-system-x86_64 -enable-kvm "$@"

Q. 請測試有無啟動 kvm，Alpine 系統開機速度差多少？

A. 有用Kvm

sudo qemu-system-x86_64 -enable-kvm -m 2048 \
-hda disk/alpine.qcow2 \
-nographic

沒加kvm

sudo qemu-system-x86_64 -m 2048 \
-hda disk/alpine.qcow2 \
-nographic

電腦基本元件

4大元件：cpu 硬碟記憶體主機板伺服器也是一台電腦，只是規格比較好，他也會有以上四大元件

cpu

中央處理器，一台電腦要去跑程式，都是cpu(中央處理器)的功勞兩大cpu廠商：Intel 和 AMD 我們現在比較的是x86架構，手機的cpu處理器不列入比較

由上往下依序，是入門款到高階

Core i7 - 11 700 k
系列 世代 性能 尾碼

AMD的處理器都能超頻

以i7-8700為例，https://www.intel.com.tw/content/www/tw/zh/products/sku/126686/intel-core-i78700-processor-12m-cache-up-to-4-60-ghz/specifications.html

如何報cpu規格： i7-8700，6核12執行序，（3.20GHz）時脈

尾數Ｕ，代表就是筆電的處理器

伺服器的（CPU）因為企業裡面幾乎都是intel，所以不討論ＡＭＤ命名（2017年以前）

2018後，怎麼看規格

看內部結構真正在跑運算process的是core不是threads，除非程式裡面有寫

但是只要稱得上虛擬化技術的都是跑threads

甚麼是vCPU？直接當threads看

記憶體

記憶體的重點，容量和速度雙通道的效能會大於單通道，也就是說兩條4G的記憶體效能會大於一條8G 規格DDR5 > DDR4 > DDR3

硬碟

在規格相同的情況下，M.2 > SSD > HDD

設定KVM Disk

擴充 alpine 硬碟容量

$ qemu-img resize disk/alpine.qcow2 +10G
Image resized.

檢查 alpine 硬碟容量

$ qemu-img info disk/alpine.qcow2
image: disk/alpine.qcow2
file format: qcow2
virtual size: 20 GiB (21474836480 bytes)
disk size: 9.75 GiB

將alpine開機

$ sudo kvm \
-m 2048 \
-hda disk/alpine.qcow2 \
-nographic

延伸空間

alpine66:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
fd0 2:0 1 0B 0 disk
sda 8:0 0 20G 0 disk
├─sda1 8:1 0 100M 0 part /boot (開機使用)
├─sda2 8:2 0 2.5G 0 part [SWAP] （）
└─sda3 8:3 0 7.4G 0 part /
sr0 11:0 1 1024M 0 rom

在業界，是用多少硬碟給多少，不會一次到位，因為給了就沒辦法壓縮回來

在實物上，windows完全不建議一個磁碟分成兩個分割區

主機板（MB）

規格分3種，體積由大到小ATX »> MICRO ATX »> MINI ATX

認識企業伺服器

vmdk是vmware 虛擬機器的硬碟檔 VHDX是微軟 Microsoft Hyper-V虛擬機器的硬碟檔 VPC是雅馬遜Amazon 虛擬機器的硬碟檔 QCOW2是KVM 虛擬機器的硬碟檔

KVM Bridge Network

學習目標

QEMU 網路設定
QEMU User Mode 網路
TUN/TAP 虛擬網卡
自製橋接網路 (Bridge)
dnsmasq DHCP Server 架設
KVM 虛擬機器透過 NAT 上網

QEMU 網路設定

確認 QEMU 支援網卡型號

*預設為 E1000 (Intel® 82540EM)

$ kvm -net nic,model=?

確認 QEMU 支援網路類型

$ kvm --help | grep "\\-net "
-net nic[,macaddr=mac][,model=type][,name=str][,addr=str][,vectors=v]
-net [user|tap|bridge|socket][,option][,option][,...]

QEMU User Mode 網路

SLIRP主要是模擬撥接網路，對應在QEMU的USER Mode 就會非常慢

User Networking (SLIRP)

there is a lot of overhead so the performance is poor
in general, ICMP traffic does not work (so you cannot use ping within a guest)
on Linux hosts, ping does work from within the guest, but it needs initial setup by root (once per host)
the guest is not directly accessible from the host or the external network

實作

先確定工作目錄

$ pwd
/home/desktop/wk

下載 Tiny Core Linux 光碟片

$ wget http://www.tinycorelinux.net/13.x/x86/release/Core-current.iso -O iso/tcl.iso

透過KVM開啟Tinycore Linux虛擬機器

kvm -m 128 -net nic -net user -cdrom iso/tcl.iso -curses

-curses 指的是不要圖形化介面 -net nic 是網卡，這邊不指定 -net user 是網路要user mode

確認IP : 10.0.2.15 和Mac address ： 52:54:00:12:34:56

＃ ifconfig

確認Gateway : 10.0.2.2

# route -n

確認DNS : 10.0.2.3

# cat /etc/resolv.conf

Tiny Core Linux 簡稱TCL，在TCL 虛擬機中確認網卡型號

安裝 lshw 套件

$ tce-load -wi lshw.tcz

確認網卡型號

$ sudo lshw -class network
::
product: 82540EM Gigabit Ethernet Controller

TCL 關機

$ sudo poweroff

TUN/TAP 虛擬網卡

TAP (as in network tap) simulates a link layer device and it operates with layer 2 packets such as Ethernet frames. TUN (as in network TUNnel) simulates a network layer device and it operates with layer 3 packets such as IP packets. TAP is used to create a network bridge, while TUN is used with routing.

TAP這張網卡主要模擬乙太網路的網卡，TAP通常用來做橋接網路

使用 tunctl 指令

安裝tunctl 套件

$ sudo apt install uml-utilities

產生 TAP 網路裝置

$ sudo tunctl -u ${USER}
Set 'tap0' persistent and owned by uid 1000

手動設定 TAP 網卡的 MAC 位址

$ sudo ifconfig tap0 hw ether 4c:22:d0:b8:78:ae

查看 TAP 網路裝置

$ ifconfig tap0
tap0: flags=4098<BROADCAST,MULTICAST> mtu 1500
ether 4c:22:d0:b8:78:ae txqueuelen 1000 (Ethernet)
::

移除 TAP 網路裝置

$ tunctl -d tap0
Set 'tap0' nonpersistent

Ethernet 重點回顧

有線乙太網路 Ethernet

一個乙太網路的要件包括了

乙太網路卡 (Ethernet card)
網路連接設備
連接設備的電纜 (cabling)

TCP/IP Model

Data-Link Layer 資料連接層 Physical Layer 實體層

Bridge & STP

A network bridge is a computer networking device that creates a single, aggregate network from multiple communication networks or network segments. In the OSI model, bridging is performed in the data link layer (layer 2).

The Spanning Tree Protocol (STP) is a network protocol that builds a loop-free logical topology for Ethernet networks. The basic function of STP is to prevent bridge loops (迴路) and the broadcast radiation (廣播風暴) that results from them.

使用brctl 指令

安裝brctl 套件

$ sudo apt install bridge-utils

產生 bridge 網路裝置

$ sudo brctl addbr br01

查看 br01 的資訊

$ brctl show br01
bridge name bridge id STP enabled interfaces
br01 8000.000000000000 no
*8000 是 id，句點後是橋接裝置的 MAC 位址

br01 裝置的內部網卡(網卡名稱與 Bridge 網路裝置同名)

$ ifconfig br01
br01: flags=4098<BROADCAST,MULTICAST> mtu 1500
ether 7e:49:b0:80:e9:62 txqueuelen 1000 (Ethernet)

移除 br01 裝置

$ sudo brctl delbr br01

自製 KVM橋接網路

設定 macsw 裝置的 IP

$ sudo ifconfig macsw 172.16.20.254 netmask 255.255.255.0 up

檢查

$ ifconfig macsw
macsw: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.16.20.254 netmask 255.255.255.0 broadcast 172.16.20.255
inet6 fe80::303b:1dff:fe2a:2324 prefixlen 64 scopeid 0x20<link>
ether 32:3b:1d:2a:23:24 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 37 bytes 4880 (4.8 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

產生二個 TAP 網路介面

$ sudo tunctl -b -u ${USER}
Tap0
$ sudo tunctl -b -u ${USER}
tap1

啟動新建 TAP 網路裝置

$ sudo ifconfig tap0 up
$ ifconfig tap0
$ sudo ifconfig tap1 up
$ ifconfig tap1

將 TAP 網路介面連接至 Bridge 網路裝置

$ sudo brctl addif macsw tap0
$ sudo brctl addif macsw tap1

檢視 macsw 網路裝置的 MAC 表格

$ sudo brctl showmacs macsw
port no mac addr is local? ageing timer
2 f6:af:90:76:95:ea yes 0.00
2 f6:af:90:76:95:ea yes 0.00
1 fa:47:ee:b7:e2:8b yes 0.00
1 fa:47:ee:b7:e2:8b yes 0.00
$ ifconfig macsw

檢視 macsw 內部運作資訊

$ sudo brctl showstp macsw

顯示所有 bridge 網路裝置

$ brctl show
bridge name bridge id STP enabled interfaces
macsw 8000.f6af907695ea no tap0
tap1

dnsmasq 介紹

dnsmasq is free software providing Domain Name System (DNS) caching, a Dynamic Host Configuration Protocol (DHCP) server, router advertisement and network boot features, intended for small computer networks.

dnsmasq has low requirements for system resources, can run on Linux, BSDs, Android and macOS, and is included in most Linux distributions. Consequently, it “is present in a lot of home routers and certain Internet of Things gadgets” and is included in Android.

檢查 53 Port 被誰佔用

$ sudo lsof -i :53
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
systemd-r 468 systemd-resolve 12u IPv4 17221 0t0 UDP localhost:domain
systemd-r 468 systemd-resolve 13u IPv4 17222 0t0 TCP localhost:domain (LISTEN)

停止並停用 systemd-resolved

$ sudo systemctl stop systemd-resolved
$ sudo systemctl disable systemd-resolved
sudo: unable to resolve host mykvm: Temporary failure in name resolution ##注意這行要sudo nano /etc/hosts進去修改
Removed /etc/systemd/system/multi-user.target.wants/systemd-resolved.service.
Removed /etc/systemd/system/dbus-org.freedesktop.resolve1.service.

在/etc/hosts 新增hosts name

$ sudo nano /etc/hosts
sudo: unable to resolve host mykvm: Temporary failure in name resolution
將127.0.0.1 localhost修改成127.0.0.1 localhost mykvm

ctrl o 儲存 ctrl x離開之後，在下sudo命令就不會噴錯了

查看 /etc/resolv.conf

$ ls -al /etc/resolv.conf
lrwxrwxrwx 1 root root 39 Apr 19 19:32 /etc/resolv.conf -> ../run/systemd/resolve/stub-resolv.conf

刪除 /etc/resolv.conf

$ sudo rm /etc/resolv.conf

產生新的 /etc/resolv.conf

$ echo nameserver 8.8.8.8 | sudo tee /etc/resolv.conf
nameserver 8.8.8.8

安裝 dnsmasq 套件

$ sudo apt install dnsmasq

編輯 dnsmasq 設定，在最下方加入下列內容

$ echo "
dhcp-range=172.16.20.150,172.16.20.160,24h
dhcp-option=option:netmask,255.255.255.0
dhcp-option=option:router,172.16.20.254
dhcp-option=option:dns-server,8.8.8.8
" | sudo tee -a /etc/dnsmasq.conf

第一行，給一個IP的範圍，讓dhcp server派發． 24h 是IP租用時間，意思是說會給client端24小時的時間，當時間經過一半，client端會去跟server續約，如果沒續約的話，server端會收回IP

重啟 dnsmasq

$ sudo systemctl restart dnsmasq

在另一個終端機啟動第一部 TCL 虛擬電腦

$ kvm -m 128 -cdrom iso/tcl.iso \
-net nic,macaddr=52:54:72:16:20:10 \
-net tap,ifname=tap0 -curses

進入 TCL 虛擬主機後請檢查

是否有自動取得IP

tc@box:~$ ifconfig
eth0 Link encap:Ethernet HWaddr 52:54:72:16:20:10
inet addr:172.16.20.150 Bcast:172.16.20.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:8 errors:0 dropped:0 overruns:0 frame:0
TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1328 (1.2 KiB) TX bytes:1086 (1.0 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

Default Gateway是否設定正確

tc@box:~$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 172.16.20.254 0.0.0.0 UG 0 0 0 eth0
127.0.0.1 0.0.0.0 255.255.255.255 UH 0 0 0 lo
172.16.20.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0

DNS是否設定正確

tc@box:~$ cat /etc/resolv.conf
nameserver 8.8.8.8

啟動第二部 TCL 虛擬電腦 $ kvm -m 128 -cdrom iso/tcl.iso
-net nic,macaddr=52:54:72:16:20:11
-net tap,ifname=tap1 -curses

是否有自動取得IP

tc@box:~$ ifconfig
eth0 Link encap:Ethernet HWaddr 52:54:72:16:20:11
inet addr:172.16.20.151 Bcast:172.16.20.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:7 errors:0 dropped:0 overruns:0 frame:0
TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1268 (1.2 KiB) TX bytes:1026 (1.0 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

Default Gateway是否設定正確

tc@box:~$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 172.16.20.254 0.0.0.0 UG 0 0 0 eth0
127.0.0.1 0.0.0.0 255.255.255.255 UH 0 0 0 lo
172.16.20.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0

DNS是否設定正確

tc@box:~$ cat /etc/resolv.conf
nameserver 8.8.8.8

企業IP網路規劃分法

IP 0~~255 去頭去尾 1~~254 前十後五中間切一刀前半靜態後半 DHCP 頭：Network ID 尾：廣播位址前10：系統服務（DNS Server , DHCP Server …等）後五：網路設備切一半前半：手動設定靜態IP 後半：DHCP Server自動配發IP

ARP 重點回顧

當Host A要傳給Host B資料時，第一步Host A主機會先檢查自己的ARP cache有沒有Host B的MAC位址，如果沒有紀錄，

它就會進到第二部，在Switch上面廣播"ARP requset"，ARP requset裡面會有：

Host A自己的IP address 和 MAC address
Host B的IP address和MAC address，但因為這時候不知道，所以顯示都是00:00:00:00:00:00（ 00:00:00:00:00:00 ）

第三部 Host B主機收到ARP request後，會在自己這邊的ARP cache紀錄Host A主機的MAC Address

第四部 Host B主機會將自己的IP位址和MAC address位址傳給Host A

第五部 Host A主機會在自己的ARP Cache中把收到的Host B 主機的IP位址和MAC address紀錄起來

清除ARP Cache

[TCL 主機 172.16.20.150]

$ arp -n
? (172.16.20.254) at 06:6f:48:e9:86:1b \[ether\] on eth0
? (172.16.20.151) at 52:54:72:16:20:11 \[ether\] on eth0

清除 ARP Cache

$ sudo arp -d 172.16.20.151
$ arp -n
? (172.16.20.254) at 06:6f:48:e9:86:1b \[ether\] on eth0

擷取封包

[HOST 主機]

$ pwd
/home/qazstuser01/wk
$ sudo tcpdump -s 0 -i macsw8 -w tcpdump.pcap
tcpdump: listening on macsw8, link-type EN10MB (Ethernet), capture size 262144 bytes

[TCL 主機 172.16.20.150]

$ ping -c 4 172.16.20.151

[HOST 主機]

回到正在執行 tcpdump 的終端機，按下 Ctrl + C 停止擷取