Skip to content

antony@notes:~/rancher$ cat "Harvester-使用-Velero-外部-CSI-儲存備份-如何確保-VM-資料一致性.md"

Harvester 使用 Velero 外部 CSI 儲存備份：如何確保 VM 資料一致性。

2026-02-28· rancher

Harvester 使用 Velero 外部 CSI 儲存備份，並確保 VM 資料一致性

1. Preface

Harvester 1.5 導入了對使用外部容器儲存介面（CSI）驅動程式來配置虛擬機器根磁碟區（root volumes）與資料磁碟區（data volumes）的支援。

本文將示範如何使用 Velero 1.17.0 在 Harvester 中執行虛擬機器的備份與還原。文章將透過具體的指令與資源清單（manifests）來完成以下操作：

  • 備份特 namespace 內的虛擬機器、其 NFS CSI 磁碟區，以及相關的命名空間層級設定檔
  • 將備份產物匯出至 AWS S3 儲存體（bucket）
  • 還原至同一個叢集（cluster）上的不同 namespace
  • 還原至不同的叢集

Velero 是一款 Kubernetes 原生的備份與還原工具，讓使用者能夠對虛擬機器執行排程與隨選（on-demand）備份，並將資料儲存至外部的物件儲存供應商（例如 S3、Azure Blob 或 GCS），完美契合企業級的備份與災難復原實務標準。

2. 安裝與部署 Garage（輕量級 S3 相容的物件儲存系統）

  1. 建立永存目錄

    sudo mkdir -p /garage/{data,meta,config}

  2. 撰寫 garage.toml 設定檔

    • [s3_api] 和 [s3_web] 區塊底下的 root_domain 的值需要根據環境調整

      cat <<EOF | sudo tee /garage/config/garage.toml > /dev/null
metadata_dir = "/var/lib/garage/meta"
data_dir = "/var/lib/garage/data"
db_engine = "sqlite"

replication_factor = 1

rpc_bind_addr = "[::]:3901"
rpc_public_addr = "127.0.0.1:3901"
rpc_secret = "ba7861f27fccfe06a99245b49e33113ca846cecdc9956169a47d7eecd74aa7d8"

[s3_api]
s3_region = "garage"
api_bind_addr = "[::]:3900"
root_domain = "api.garage.kubeantony.com"

[s3_web]
bind_addr = "[::]:3902"
root_domain = "web.garage.kubeantony.com"
index = "index.html"

[k2v_api]
api_bind_addr = "[::]:3904"

[admin]
api_bind_addr = "[::]:3903"
admin_token = "KV0EuVoDxze0Ut7tQdkvNiU8TKcD8LejZQglH/JGK00="
metrics_token = "RQxx1DjKcnhpEVnPmjxWn+CzLY5wNBNT60GMvHQ+LXo="
EOF

  3. 透過 Podman 部署 Garage

    sudo podman run \
  -d \
  --name garaged \
  --network host \
  -v /garage/config/garage.toml:/etc/garage.toml \
  -v /garage/meta:/var/lib/garage/meta \
  -v /garage/data:/var/lib/garage/data \
  dxflrs/garage:v2.2.0

  4. 確認 Garage Container 運作狀態

    sudo podman ps -a --filter name=garage

    執行結果：

    CONTAINER ID  IMAGE                           COMMAND         CREATED         STATUS         PORTS       NAMES
499fff9ed2db  docker.io/dxflrs/garage:v2.2.0  /garage server  31 minutes ago  Up 31 minutes              garaged

  5. 查詢 Garage 節點狀態

    sudo podman exec -it garaged /garage status

    執行結果：

    2026-02-26T06:41:45.072105Z  INFO garage_net::netapp: Connected to 127.0.0.1:3901, negotiating handshake...
2026-02-26T06:41:45.113726Z  INFO garage_net::netapp: Connection established to 368849c86cba2382
==== HEALTHY NODES ====
ID                Hostname               Address         Tags  Zone  Capacity          DataAvail  Version
368849c86cba2382  garage.kubeantony.com  127.0.0.1:3901              NO ROLE ASSIGNED             v2.2.0

  6. 將指定的 Garage 節點設定至 dc1 可用區（Zone）和 50GB 的儲存容量。

    sudo podman exec -it garaged /garage layout assign -z dc1 -c 50G 368849c86cba2382

    執行結果：

    2026-02-26T06:42:30.841741Z  INFO garage_net::netapp: Connected to 127.0.0.1:3901, negotiating handshake...
2026-02-26T06:42:30.882712Z  INFO garage_net::netapp: Connection established to 368849c86cba2382
Role changes are staged but not yet committed.
Use `garage layout show` to view staged role changes,
and `garage layout apply` to enact staged changes.

    執行結果顯示此變更目前僅處於「暫存 staged」狀態

  7. 正式套用先前暫存的叢集配置（版本 1），使節點的角色與容量設定正式生效。

    sudo podman exec garaged /garage layout apply --version 1

    執行結果：

    2026-02-26T06:42:50.085913Z  INFO garage_net::netapp: Connected to 127.0.0.1:3901, negotiating handshake...
2026-02-26T06:42:50.126651Z  INFO garage_net::netapp: Connection established to 368849c86cba2382
==== COMPUTATION OF A NEW PARTITION ASSIGNATION ====

Partitions are replicated 1 times on at least 1 distinct zones.

Optimal partition size:                     195.3 MB
Usable capacity / total cluster capacity:   50.0 GB / 50.0 GB (100.0 %)
Effective capacity (replication factor 1):  50.0 GB

dc1                 Tags  Partitions        Capacity  Usable capacity
  368849c86cba2382  []    256 (256 new)     50.0 GB   50.0 GB (100.0%)
  TOTAL                   256 (256 unique)  50.0 GB   50.0 GB (100.0%)


New cluster layout with updated role assignment has been applied in cluster.
Data will now be moved around between nodes accordingly.

  8. 再次確認 Garage 節點狀態

    sudo podman exec garaged /garage status

    執行結果：

    2026-02-26T07:49:07.462651Z  INFO garage_net::netapp: Connected to 127.0.0.1:3901, negotiating handshake...
2026-02-26T07:49:07.503706Z  INFO garage_net::netapp: Connection established to 368849c86cba2382
==== HEALTHY NODES ====
ID                Hostname               Address         Tags  Zone  Capacity  DataAvail        Version
368849c86cba2382  garage.kubeantony.com  127.0.0.1:3901  []    dc1   50.0 GB   58.5 GB (82.0%)  v2.2.0

  9. 建立 Bucket

    sudo podman exec garaged /garage bucket create vm-backup

    執行結果：

    2026-02-26T07:54:42.568429Z  INFO garage_net::netapp: Connected to 127.0.0.1:3901, negotiating handshake...
2026-02-26T07:54:42.609720Z  INFO garage_net::netapp: Connection established to 368849c86cba2382
==== BUCKET INFORMATION ====
Bucket:          be30df62bea032acdc9ba3b992b32a43794b9f950599870e21ee706340c39c96
Created:         2026-02-26 07:54:42.610 +00:00

Size:            0 B (0 B)
Objects:         0

Website access:  false

Global alias:    vm-backup

==== KEYS FOR THIS BUCKET ====
Permissions  Access key    Local aliases

  10. 建立 API Key

    sudo podman exec garaged /garage key create vm-backup-access-key

    執行結果：

    2026-02-26T07:56:11.307951Z  INFO garage_net::netapp: Connected to 127.0.0.1:3901, negotiating handshake...
2026-02-26T07:56:11.348732Z  INFO garage_net::netapp: Connection established to 368849c86cba2382
==== ACCESS KEY INFORMATION ====
Key ID:              GK6e4243deca7bc09fdffcb405
Key name:            vm-backup-access-key
Secret key:          9280dbc88cb656a8589783f0762652fdb93362b2df0c7a47cfa7f2ae9b15899f
Created:             2026-02-26 07:56:11.348 +00:00
Validity:            valid
Expiration:          never

Can create buckets:  false

==== BUCKETS FOR THIS KEY ====
Permissions  ID  Global aliases  Local aliases

  11. 賦予權限

    sudo podman exec garaged /garage bucket allow --read --write --owner vm-backup --key vm-backup-access-key

    執行結果：

    2026-02-26T07:57:44.077279Z  INFO garage_net::netapp: Connected to 127.0.0.1:3901, negotiating handshake...
2026-02-26T07:57:44.118727Z  INFO garage_net::netapp: Connection established to 368849c86cba2382
==== BUCKET INFORMATION ====
Bucket:          be30df62bea032acdc9ba3b992b32a43794b9f950599870e21ee706340c39c96
Created:         2026-02-26 07:54:42.610 +00:00

Size:            0 B (0 B)
Objects:         0

Website access:  false

Global alias:    vm-backup

==== KEYS FOR THIS BUCKET ====
Permissions  Access key                                        Local aliases
RWO          GK6e4243deca7bc09fdffcb405  vm-backup-access-key

  12. 測試掛載，安裝 s3fs

    sudo apt-get install s3fs -y

  13. 設定 s3fs

    echo "GK6e4243deca7bc09fdffcb405:9280dbc88cb656a8589783f0762652fdb93362b2df0c7a47cfa7f2ae9b15899f" > ${HOME}/.passwd-s3fs
chmod 600 ${HOME}/.passwd-s3fs
sudo sed -i 's/#user_allow_other/user_allow_other/' /etc/fuse.conf

  14. 建立掛載點

    sudo mkdir -p /mnt/garage-s3
sudo chown $USER:$USER /mnt/garage-s3

  15. 將自建 Garage S3 上的 vm-backup bucket 儲存體掛載至本地 /mnt/garage-s3

    s3fs vm-backup /mnt/garage-s3 \
    -o passwd_file=${HOME}/.passwd-s3fs \
    -o url=http://garage.kubeantony.com:3900 \
    -o use_path_request_style \
    -o allow_other \
    -o umask=000 \
    -o endpoint=garage

  16. 檢視 /mnt/garage-s3 目錄已掛載到 s3

    findmnt -t fuse.s3fs

    執行結果：

    TARGET         SOURCE
                    FSTYPE    OPTIONS
/mnt/garage-s3 s3fs fuse.s3fs rw,nosuid,nodev,relatime,user_id=1000,group_id=1000,allow_other

  17. 寫入資料

    echo "test connection" > /mnt/garage-s3/test.txt

  18. 查詢 vm-backup Bucket 詳細狀態

    sudo podman exec garaged /garage bucket info vm-backup

    執行結果：

    2026-02-26T08:17:07.919852Z  INFO garage_net::netapp: Connected to 127.0.0.1:3901, negotiating handshake...
2026-02-26T08:17:07.960680Z  INFO garage_net::netapp: Connection established to 368849c86cba2382
==== BUCKET INFORMATION ====
Bucket:          be30df62bea032acdc9ba3b992b32a43794b9f950599870e21ee706340c39c96
Created:         2026-02-26 07:54:42.610 +00:00

Size:            16 B (16 B)
Objects:         1

Website access:  false

Global alias:    vm-backup

==== KEYS FOR THIS BUCKET ====
Permissions  Access key                                        Local aliases
RWO          GK6e4243deca7bc09fdffcb405  vm-backup-access-key

  19. 清除測試資料與取消掛載

    rm /mnt/garage-s3/test.txt
umount /mnt/garage-s3

3. Install and Configure Velero

  1. ssh 連線到 Harvester Control plane 節點執行以下操作

  2. 安裝 Velero CLI

    # 設定版本
VERSION=v1.17.0

# 使用 curl 下載並重新命名檔案
curl -L https://github.com/vmware-tanzu/velero/releases/download/$VERSION/velero-$VERSION-linux-amd64.tar.gz -o velero.tar.gz

# 解壓縮
tar -xvf velero.tar.gz

# 移動到系統路徑
mv velero-$VERSION-linux-amd64/velero /root/bin/

# 測試
velero version --client-only

    執行結果：

    Client:
        Version: v1.17.0
        Git commit: 3172d9f99c3d501aad9ddfac8176d783f7692dce

  3. 設置以下 shell 變數

    BUCKET_NAME=vm-backup
BUCKET_REGION=garage
echo "[default]
aws_access_key_id=GK6e4243deca7bc09fdffcb405
aws_secret_access_key=9280dbc88cb656a8589783f0762652fdb93362b2df0c7a47cfa7f2ae9b15899f" > ${HOME}/.passwd
AWS_CREDENTIALS_FILE=${HOME}/.passwd

  4. 在 Harvester 集群上安裝 Velero

    velero install \
  --provider aws \
  --features=EnableCSI \
  --plugins "velero/velero-plugin-for-aws:v1.12.0,quay.io/kubevirt/kubevirt-velero-plugin:v0.7.1" \
  --bucket "${BUCKET_NAME}" \
  --secret-file "${AWS_CREDENTIALS_FILE}" \
  --backup-location-config region="${BUCKET_REGION}" \
  --snapshot-location-config region="${BUCKET_REGION}" \
  --use-node-agent

    在這個設定中，Velero 被設定為：

    • 跑在 velero namespace
    • 啟用 CSI 儲存卷快照 (volume snapshot) API
    • 啟用內建的節點代理程式 (node agent) 資料移動控制器與 Pod
    • 使用 velero-plugin-for-aws 外掛程式來管理與 S3 物件儲存庫的互動
    • 使用 kubevirt-velero-plugin 外掛程式來備份與還原 KubeVirt 資源

  5. 確認 Velero 已安裝並正在運行

    kubectl -n velero get pods

    執行結果：

    NAME                      READY   STATUS    RESTARTS   AGE
node-agent-49f56          1/1     Running   0          4m30s
node-agent-dhtmj          1/1     Running   0          4m30s
node-agent-n4wh9          1/1     Running   0          4m30s
velero-6c57f94f49-bxcjx   1/1     Running   0          4m30s

  6. 設定 velero CLI 以輸出 CSI 物件的備份和還原狀態

    velero client config set features=EnableCSI

  7. patch backupstoragelocation

    kubectl patch backupstoragelocation default -n velero --type merge --patch '{"spec":{"config":{"s3Url":"http://garage.kubeantony.com:3900","s3ForcePathStyle":"true"}}}'

  8. Check Backup-location status

    velero backup-location get

    執行結果:

    NAME      PROVIDER   BUCKET/PREFIX   PHASE       LAST VALIDATED                  ACCESS MODE   DEFAULT
default   aws        vm-backup       Available   2026-02-28 11:56:20 +0000 UTC   ReadWrite     true

4. Deploy the NFS CSI and Example Server

  1. 在 Kubernetes 叢集上設置 NFS 伺服器

    curl -s -O https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/deploy/example/nfs-provisioner/nfs-server.yaml
vim nfs-server.yaml

    檔案內容：

          volumes:
        - name: nfs-vol
          hostPath:
            path: /root/nfs-data  # modify this to specify another path to store nfs share data
            type: DirectoryOrCreate

  2. 部署 NFS 伺服器

    kubectl apply -f nfs-server.yaml

    執行結果：

    service/nfs-server created
deployment.apps/nfs-server created

  3. 檢查 NFS 伺服器是否正常運作

    kubectl get pods

    執行結果：

    NAME                          READY   STATUS    RESTARTS   AGE
nfs-server-69ff4786bd-4pf6v   1/1     Running   0          3m22s

  4. 部署 NFS CSI 驅動程式

    curl -skSL https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/v4.13.1/deploy/install-driver.sh | bash -s v4.13.1 --

    執行結果：

    Installing NFS CSI driver, version: v4.13.1 ...
serviceaccount/csi-nfs-controller-sa created
serviceaccount/csi-nfs-node-sa created
clusterrole.rbac.authorization.k8s.io/nfs-external-provisioner-role created
clusterrolebinding.rbac.authorization.k8s.io/nfs-csi-provisioner-binding created
clusterrole.rbac.authorization.k8s.io/nfs-external-resizer-role created
clusterrolebinding.rbac.authorization.k8s.io/nfs-csi-resizer-role created
csidriver.storage.k8s.io/nfs.csi.k8s.io created
deployment.apps/csi-nfs-controller created
daemonset.apps/csi-nfs-node created
NFS CSI driver installed successfully.

  5. 確認 pods 狀態

    kubectl -n kube-system get pod -o wide -l 'app in (csi-nfs-controller, csi-nfs-node)'

    執行結果：

    NAME                                  READY   STATUS    RESTARTS   AGE     IP               NODE     NOMINATED NODE   READINESS GATES
csi-nfs-controller-68599bd8b6-v68cr   5/5     Running   0          8m36s   192.168.11.183   hvx-03   <none>           <none>
csi-nfs-node-2s57l                    3/3     Running   0          8m35s   192.168.11.181   hvx-01   <none>           <none>
csi-nfs-node-79zpf                    3/3     Running   0          8m35s   192.168.11.182   hvx-02   <none>           <none>
csi-nfs-node-smcq8                    3/3     Running   0          8m35s   192.168.11.183   hvx-03   <none>           <none>

  6. 建立 Client pod 掛載 NFS Volume 檢查 NFS 伺服器是否正常運作

    kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/deploy/example/nfs-provisioner/nginx-pod.yaml

  7. 確認 client pod 運作正常

    kubectl get pod nginx-nfs-example

    執行結果：

    NAME                READY   STATUS    RESTARTS   AGE
nginx-nfs-example   1/1     Running   0          43m

  8. 從範例 pod 檢查掛載點

    kubectl exec nginx-nfs-example -- bash -c "findmnt /var/www -o TARGET,SOURCE,FSTYPE"

    執行結果：

    TARGET   SOURCE                                 FSTYPE
/var/www nfs-server.default.svc.cluster.local:/ nfs4

  9. 清除 client pod

    kubectl delete -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/deploy/example/nfs-provisioner/nginx-pod.yaml

  10. 撰寫 NFS Storage Class YAML 檔

    vi nfs-sc.yaml

    檔案內容如下:

    apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-csi
provisioner: nfs.csi.k8s.io
parameters:
  server: nfs-server.default.svc.cluster.local
  share: /
  # csi.storage.k8s.io/provisioner-secret is only needed for providing mountOptions in DeleteVolume
  # csi.storage.k8s.io/provisioner-secret-name: "mount-options"
  # csi.storage.k8s.io/provisioner-secret-namespace: "default"
reclaimPolicy: Delete
volumeBindingMode: Immediate
allowVolumeExpansion: true
mountOptions:
  - nfsvers=4.1

  11. 建立 NFS Storage Class

    kubectl apply -f nfs-sc.yaml

  12. Create source PVC and an example pod to write data

    kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/deploy/example/pvc-nfs-csi-dynamic.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/deploy/example/nginx-pod-nfs.yaml

  13. Check the Source PVC

    kubectl exec nginx-nfs -- ls /mnt/nfs

    執行結果:

    outfile

  14. Create a snapshot on source PVC

    kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/deploy/example/snapshot/snapshotclass-nfs.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/deploy/example/snapshot/snapshot-nfs-dynamic.yaml

  15. Check snapshot Status

    kubectl describe volumesnapshot test-nfs-snapshot

    執行結果:

    Name:         test-nfs-snapshot
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  snapshot.storage.k8s.io/v1
Kind:         VolumeSnapshot
Metadata:
  Creation Timestamp:  2026-02-28T09:37:06Z
  Finalizers:
    snapshot.storage.kubernetes.io/volumesnapshot-as-source-protection
    snapshot.storage.kubernetes.io/volumesnapshot-bound-protection
  Generation:        1
  Resource Version:  15038120
  UID:               59a4ddb9-a123-4dea-b220-e0fcac477f59
Spec:
  Source:
    Persistent Volume Claim Name:  pvc-nfs-dynamic
  Volume Snapshot Class Name:      csi-nfs-snapclass
Status:
  Bound Volume Snapshot Content Name:  snapcontent-59a4ddb9-a123-4dea-b220-e0fcac477f59
  Creation Time:                       2026-02-28T09:37:06Z
  Ready To Use:                        true
  Restore Size:                        1026
Events:
  Type    Reason            Age   From                 Message
  ----    ------            ----  ----                 -------
  Normal  CreatingSnapshot  46s   snapshot-controller  Waiting for a snapshot default/test-nfs-snapshot to be created by the CSI driver.
  Normal  SnapshotCreated   46s   snapshot-controller  Snapshot default/test-nfs-snapshot was successfully created by the CSI driver.
  Normal  SnapshotReady     46s   snapshot-controller  Snapshot default/test-nfs-snapshot is ready to use.

    In above example, snapcontent-59a4ddb9-a123-4dea-b220-e0fcac477f59 is the snapshot name.

  16. Create a new PVC based on snapshot

    kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/deploy/example/snapshot/pvc-nfs-snapshot-restored.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/deploy/example/snapshot/nginx-pod-restored-snapshot.yaml

  17. Check data

    kubectl exec nginx-nfs-restored-snapshot -- ls /mnt/nfs

    執行結果:

    outfile

  18. 清除範例 pod

    kubectl delete -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/deploy/example/nginx-pod-nfs.yaml
kubectl delete -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/deploy/example/pvc-nfs-csi-dynamic.yaml

kubectl delete -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/deploy/example/snapshot/nginx-pod-restored-snapshot.yaml
kubectl delete -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/deploy/example/snapshot/pvc-nfs-snapshot-restored.yaml

5. Preparing the Virtual Machine and Image

  1. Create a custom namespace named demo-src

    kubectl create ns demo-src

  2. 上傳 ubuntu images

    https://cloud-images.ubuntu.com/minimal/releases/noble/release-20260218/ubuntu-24.04-minimal-cloudimg-amd64.img

    點選左側選單 images -> 在 URL 欄位輸入以上網址

    image

    Stoage 選擇 nfs-csi -> Create image

    確認虛擬機映像檔已成功上傳至 Harvester： image

  3. 建立 VM

    輸入 VM 名稱: test

    image

    點選 Volumes -> 選擇剛剛上傳的 Image demo-src/ubuntu-24.04-minimal-cloudimg-amd64.img (nfs-csi / 3.5 Gi) image

    點選 Add Volume -> StorageClass nfs-csi -> Volume Mode: Filesystem

    image

    Cloud Config user-data

    ssh_pwauth: true

users:
  - default
  - name: rancher
    gecos: rancher
    groups: users
    lock_passwd: false
    sudo: ALL=(ALL) NOPASSWD:ALL
    plain_text_passwd: 'rancher'
    shell: /bin/bash

    點選 Console -> Open in Serial Console

    image

  4. add some files to both the root and data volumes.

    sudo touch /test

sudo mkdir /data
sudo mkfs.ext4 /dev/vdb
echo '/dev/vdb /data ext4 defaults 0 2' | sudo tee -a /etc/fstab
sudo mount -a
echo "haha" | sudo tee /data/test

6. 驗證檔案系統凍結相容性

  1. Access the virtual machine’s virt-launcher compute container

    POD=$(kubectl get pods -n demo-src \
  -l vm.kubevirt.io/name=test \
  -o jsonpath='{.items[0].metadata.name}')
kubectl exec -it $POD -n demo-src -c compute -- bash

  2. Test filesystem freeze using the virt-freezer application available in the compute container:

    virt-freezer --freeze --namespace demo-src --name test

    執行結果:

    {"component":"freezer","level":"info","msg":"Guest agent version is 8.2.2","pos":"virt-freezer.go:114","timestamp":"2026-02-28T14:00:43.873324Z"}
{"component":"freezer","level":"info","msg":"Operation completed successfully","pos":"virt-freezer.go:152","timestamp":"2026-02-28T14:00:43.940305Z"}

  3. 解凍

    virt-freezer --unfreeze --namespace demo-src --name test

    執行結果:

    {"component":"freezer","level":"info","msg":"Guest agent version is 8.2.2","pos":"virt-freezer.go:114","timestamp":"2026-02-28T14:02:06.328210Z"}
{"component":"freezer","level":"info","msg":"Operation completed successfully","pos":"virt-freezer.go:152","timestamp":"2026-02-28T14:02:06.338162Z"}

  4. 設定在開始建立備份前凍結 VM 檔案系統，備份完成後解凍虛擬機檔案系統

    kubectl patch virtualmachine test -n demo-src --type merge --patch '{
  "spec": {
    "template": {
      "metadata": {
        "annotations": {
          "pre.hook.backup.velero.io/command": "[\"/usr/bin/virt-freezer\", \"--freeze\", \"--namespace\", \"demo-src\", \"--name\", \"ub-test\"]",
          "pre.hook.backup.velero.io/container": "compute",
          "pre.hook.backup.velero.io/on-error": "Fail",
          "pre.hook.backup.velero.io/timeout": "30s",
          "post.hook.backup.velero.io/command": "[\"/usr/bin/virt-freezer\", \"--thaw\", \"--namespace\", \"demo-src\", \"--name\", \"ub-test\"]",
          "post.hook.backup.velero.io/container": "compute",
          "post.hook.backup.velero.io/on-error": "Continue",
          "post.hook.backup.velero.io/timeout": "30s"
        }
      }
    }
  }
}'

  5. 到 Harvester UI 將目標 vm 重新開機，套用最新的設定檔

7. Backup the Source Namespace

  1. 使用 velero CLI 利用 Velero 內建的資料移動工具，建立 demo-src 命名空間的備份

    BACKUP_NAME=backup-demo-src-`date "+%s"`

velero backup create "${BACKUP_NAME}" \
  --include-namespaces demo-src \
  --snapshot-move-data

    執行結果:

    Backup request "backup-demo-src-1772286287" submitted successfully.
Run `velero backup describe backup-demo-src-1772286287` or `velero backup logs backup-demo-src-1772286287` for more details.

  2. DataUpload 的自訂資源提供了備份進度的見解

    velero backup get "${BACKUP_NAME}"

    執行結果:

    NAME                         STATUS       ERRORS   WARNINGS   CREATED                         EXPIRES   STORAGE LOCATION   SELECTOR
backup-demo-src-1772278089   InProgress   0        0          2026-02-28 11:56:33 +0000 UTC   29d       default            <none>

    …過幾分鐘後

    NAME                         STATUS      ERRORS   WARNINGS   CREATED                         EXPIRES   STORAGE LOCATION   SELECTOR
backup-demo-src-1772278089   Completed   0        0          2026-02-28 11:56:33 +0000 UTC   29d       default            <none>

8. Restore To A Different Namespace

  1. Save the following restore modifier to a local file named modifier-data-volumes.yaml

    cat <<EOF > modifier-data-volumes.yaml
version: v1
resourceModifierRules:
- conditions:
    groupResource: persistentvolumeclaims
    matches:
    - path: /metadata/annotations/harvesterhci.io~1volumeForVirtualMachine
      value: "\"true\""
  patches:
  - operation: remove
    path: /metadata/annotations/harvesterhci.io~1volumeForVirtualMachine
EOF

  2. Create the restore modifier

    kubectl -n velero create cm modifier-data-volumes --from-file=modifier-data-volumes.yaml

  3. Assign the backup name to a shell variable

    velero restore create \
  --from-backup "${BACKUP_NAME}" \
  --namespace-mappings "demo-src:demo-dst" \
  --exclude-resources "virtualmachineimages.harvesterhci.io" \
  --resource-modifier-configmap "modifier-data-volumes" \
  --labels "velero.kubevirt.io/clear-mac-address=true,velero.kubevirt.io/generate-new-firmware-uuid=true"

    執行結果:

    Restore request "backup-demo-src-1772286287-20260228141734" submitted successfully.
Run `velero restore describe backup-demo-src-1772286287-20260228141734` or `velero restore logs backup-demo-src-1772286287-20260228141734` for more details.

    修復期間： 虛擬機的 MAC 位址與韌體 UUID 會被重置，以避免與現有虛擬機發生潛在衝突。 虛擬機映像清單被排除，因為 Velero 會從備份中恢復整個虛擬機狀態。 會呼叫 modifier-data-volumes 的還原修飾符來修改虛擬機器的資料卷的元資料，以避免與 CDI 卷匯入填充器發生衝突。

  4. 當還原操作仍在進行中時，DataDownload 的自訂資源可用來檢視作業進度：

    RESTORE_NAME=backup-demo-src-1772286287-20260228141734

kubectl -n velero get datadownload -l velero.io/restore-name="${RESTORE_NAME}"

    執行結果:

    NAME                                              STATUS      STARTED   BYTES DONE    TOTAL BYTES   STORAGE LOCATION   AGE   NODE
backup-demo-src-1772286287-20260228135229-9fnqg   Completed   59s       3758096384    3758096384    default            75s   hvx-02
backup-demo-src-1772286287-20260228135229-fqrhx   Completed   72s       192040960     192040960     default            74s   hvx-02
backup-demo-src-1772286287-20260228135229-kp2jw   Completed   71s       10737418240   10737418240   default            75s   hvx-01
backup-demo-src-1772286287-20260228135229-n7bcp   Completed   70s       10146021376   10146021376   default            74s   hvx-03

  5. 確認還原成功完成：

    velero restore get

    執行結果:

    NAME                                        BACKUP                       STATUS      STARTED                         COMPLETED                       ERRORS   WARNINGS   CREATED                         SELECTOR
backup-demo-src-1772286287-20260228135229   backup-demo-src-1772286287   Completed   2026-02-28 13:52:29 +0000 UTC   2026-02-28 13:54:00 +0000 UTC   0        6          2026-02-28 13:52:29 +0000 UTC   <none>

  6. 確認虛擬機及其設定是否已恢復到新的 demo-dst namespace：

How to update rke2 to cis profileInstall Prometheus with Rancher Application Collection