Antony 筆記：技術原理與實務經驗 – 學習筆記

生成式 AI 課程

Fri, 07 Feb 2025 03:52:38 +0800

環境準備

Hardware

os: windows server 2025
cpu: 8 cores
MEM: 32 G
disk: 70 G

安裝軟體

Python 3.8 以上的版本 (安裝前記得勾加入 PATH) 自行挑選：https://www.python.org/downloads/ 特定版本：https://www.python.org/downloads/release/python-3119/
Cursor 或 Visual Studio Code Cursor：https://www.cursor.com/ （同時註冊帳號） https://cursor.directory/ VSCode：https://code.visualstudio.com/download
Ollama https://ollama.com/download
Docker https://www.docker.com/

以上軟體都安裝好後，

先打開 powershell 執行以下命令安裝 deepseek-r1:1.5b

ollama run deepseek-r1:1.5b

執行結果 :

pulling manifest
pulling aabd4debf0c8... 100% ▕███████████████████████████████████████████████████████▏ 1.1 GB
pulling 369ca498f347... 100% ▕███████████████████████████████████████████████████████▏ 387 B
pulling 6e4c38e1172f... 100% ▕███████████████████████████████████████████████████████▏ 1.1 KB
pulling f4d24e9138dd... 100% ▕███████████████████████████████████████████████████████▏ 148 B
pulling a85fe2a2e58e... 100% ▕███████████████████████████████████████████████████████▏ 487 B
verifying sha256 digest
writing manifest
success
>>> Send a message (/? for help)

在 docker desktop 執行以下指令

docker run -d -p 3000:8080 \
--add-host=host.docker.internal:host-gateway \
-v openwebui:/app/backend/data \
--name open-webui \
--restart always \
ghcr.io/open-webui/open-webui:main

確認 open-web-ui 運作正常

docker ps

執行結果 :

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8350f31409b2 ghcr.io/open-webui/open-webui:main "bash start.sh" About a minute ago Up About a minute (health: starting) 0.0.0.0:3000->8080/tcp open-webui

打開瀏覽器，輸入 : http://localhost:3000 連到 UI

實作 : 利用 PyTorch 建立一個 Word2Vec 詞嵌入模型

$ mkdir llm0207

打開 cursor 並進入 llm0207 專案，產生 requirements.txt 檔案 (檔名要一模一樣)

pandas
numpy
matplotlib
seaborn
scikit-learn
transformers
torch
torchvision

按 Ctrl + s 存檔，再按 Ctrl + shift + p

> Python: Create Environment
> 選擇 Venv type
> Use Existing
> 選 requirements.txt 按 ok 就會安裝

或是可以直接執行以下指令安裝 pip install -r requirements.txt

用指令檢查有沒有安裝成功

pip list

執行結果 :

Package Version
------------------ ------------
backports.tarfile 1.2.0
build 1.2.2.post1
CacheControl 0.14.2
certifi 2025.1.31
charset-normalizer 3.4.1
cleo 2.1.0
colorama 0.4.6
contourpy 1.3.1
crashtest 0.4.1
cycler 0.12.1
distlib 0.3.9
dulwich 0.22.7
fastjsonschema 2.21.1
filelock 3.17.0
fonttools 4.55.8
fsspec 2025.2.0
huggingface-hub 0.28.1
idna 3.10
importlib_metadata 8.6.1
installer 0.7.0
jaraco.classes 3.4.0
jaraco.context 6.0.1
jaraco.functools 4.1.0
Jinja2 3.1.5
joblib 1.4.2
keyring 25.6.0
kiwisolver 1.4.8
MarkupSafe 3.0.2
matplotlib 3.10.0
more-itertools 10.6.0
mpmath 1.3.0
msgpack 1.1.0
networkx 3.4.2
numpy 2.2.2
packaging 24.2
pandas 2.2.3
pillow 11.1.0
pip 25.0
pkginfo 1.12.0
platformdirs 4.3.6
poetry 2.0.1
poetry-core 2.0.1
pyparsing 3.2.1
pyproject_hooks 1.2.0
python-dateutil 2.9.0.post0
pytz 2025.1
pywin32-ctypes 0.2.3
PyYAML 6.0.2
RapidFuzz 3.12.1
regex 2024.11.6
requests 2.32.3
requests-toolbelt 1.0.0
safetensors 0.5.2
scikit-learn 1.6.1
scipy 1.15.1
seaborn 0.13.2
setuptools 65.5.0
shellingham 1.5.4
six 1.17.0
sympy 1.13.1
threadpoolctl 3.5.0
tokenizers 0.21.0
tomlkit 0.13.2
torch 2.6.0
torchvision 0.21.0
tqdm 4.67.1
transformers 4.48.2
trove-classifiers 2025.1.15.22
typing_extensions 4.12.2
tzdata 2025.1
urllib3 2.3.0
virtualenv 20.29.1
zipp 3.21.0

產生檔案 : 01_word2vec.py

01_word2vec.py
"""
Word2Vec 實作模組
使用 Skip-gram 模型實作詞向量訓練。
此模組提供基本的詞嵌入訓練功能，並包含相似詞查詢和視覺化功能。
主要功能：
1. 文本預處理
2. Skip-gram 模型訓練
3. 詞向量相似度計算
4. 詞向量視覺化
"""
# 匯入需要的套件
import torch # PyTorch 深度學習框架
import torch.nn as nn # 神經網路模組
import torch.optim as optim # 優化器模組
import numpy as np # 數值計算套件
from collections import Counter # 用於計算詞頻
import matplotlib.pyplot as plt # 繪圖套件
from sklearn.metrics.pairwise import cosine_similarity # 餘弦相似度計算
from sklearn.decomposition import PCA # 主成分分析
from typing import List, Tuple, Dict # 型別提示
# 設定 matplotlib 中文字型
plt.rcParams['font.sans-serif'] = ['Microsoft JhengHei'] # 設定Windows預設中文字型
plt.rcParams['axes.unicode_minus'] = False # 修正負號顯示問題
# 檢查系統可用的中文字型
def check_chinese_fonts():
fonts = [f.name for f in fm.fontManager.ttflist] # 取得所有可用字型
print("系統可用的中文字型：")
chinese_fonts = [f for f in fonts if ('Microsoft' in f) or ('微軟' in f) or ('Arial' in f)]
for f in chinese_fonts:
print(f)
# 定義模型訓練的超參數
WINDOW_SIZE: int = 2 # 上下文窗口大小
EMBEDDING_DIM: int = 10 # 詞向量維度
EPOCHS: int = 200 # 訓練回合數
LEARNING_RATE: float = 0.005 # 學習率
class SkipGram(nn.Module):
"""
Skip-gram 模型類別
用於學習詞向量的神經網路模型，包含一個嵌入層和一個輸出層。
"""
def __init__(self, vocab_size: int, embedding_dim: int) -> None:
"""
初始化模型架構
參數:
vocab_size: 詞彙表大小（有多少不同的詞）
embedding_dim: 詞向量的維度
"""
super(SkipGram, self).__init__()
self.embeddings = nn.Embedding(vocab_size, embedding_dim) # 詞嵌入層
self.output_layer = nn.Linear(embedding_dim, vocab_size) # 全連接輸出層
self.log_softmax = nn.LogSoftmax(dim=-1) # 對數 Softmax 激活函數
def forward(self, target: torch.Tensor) -> torch.Tensor:
"""
前向傳播函式
參數:
target: 目標詞的索引張量
回傳:
模型預測的機率分布（取對數後）
"""
embed = self.embeddings(target) # 將輸入轉換為詞向量
output = self.output_layer(embed) # 通過輸出層
return self.log_softmax(output) # 計算對數機率
def preprocess_text(corpus: str) -> Tuple[List[str], Dict[str, int], Dict[int, str]]:
"""
預處理文本資料
參數:
corpus (str): 原始文本
回傳:
Tuple[List[str], Dict[str, int], Dict[int, str]]:
- 處理後的單詞列表
- 單詞到索引的映射
- 索引到單詞的映射
"""
words = corpus.lower().replace('.', '').split()
word_counts = Counter(words)
vocab = list(word_counts.keys())
word_to_ix = {word: i for i, word in enumerate(vocab)}
ix_to_word = {i: word for word, i in word_to_ix.items()}
return words, word_to_ix, ix_to_word
def generate_training_pairs(words: List[str], window_size: int) -> List[Tuple[str, str]]:
"""
生成訓練用的詞對
參數:
words (List[str]): 單詞列表
window_size (int): 上下文窗口大小
回傳:
List[Tuple[str, str]]: 目標詞和上下文詞的配對列表
"""
pairs = []
for idx, word in enumerate(words):
for neighbor in range(-window_size, window_size + 1):
if neighbor == 0:
continue
neighbor_idx = idx + neighbor
if 0 <= neighbor_idx < len(words):
pairs.append((word, words[neighbor_idx]))
return pairs
def find_similar_words(word: str,
word_embeddings: np.ndarray,
word_to_ix: Dict[str, int],
ix_to_word: Dict[int, str],
top_n: int = 3) -> List[str]:
"""
尋找與目標詞最相似的詞
參數:
word (str): 目標詞
word_embeddings (np.ndarray): 詞向量矩陣
word_to_ix (Dict[str, int]): 單詞到索引的映射
ix_to_word (Dict[int, str]): 索引到單詞的映射
top_n (int): 要返回的相似詞數量
回傳:
List[str]: 最相似詞的列表
"""
target_vector = word_embeddings[word_to_ix[word]].reshape(1, -1)
similarities = cosine_similarity(target_vector, word_embeddings)
similar_indices = np.argsort(similarities[0])[::-1][1:top_n+1]
return [ix_to_word[i] for i in similar_indices]
def main():
"""
主程式執行流程
"""
# 建立語料文本
corpus = "I am happy because I am learning. I am excited to learn. Happy students are learning quickly."
# 資料預處理
words, word_to_ix, ix_to_word = preprocess_text(corpus)
pairs = generate_training_pairs(words, WINDOW_SIZE)
training_data = [(word_to_ix[target], word_to_ix[context]) for target, context in pairs]
# 初始化模型
vocab_size = len(word_to_ix)
model = SkipGram(vocab_size, EMBEDDING_DIM)
criterion = nn.NLLLoss()
optimizer = optim.Adam(model.parameters(), lr=LEARNING_RATE)
# 訓練模型
for epoch in range(EPOCHS):
total_loss = 0
for target, context in training_data:
target_tensor = torch.tensor([target], dtype=torch.long)
context_tensor = torch.tensor([context], dtype=torch.long)
optimizer.zero_grad()
output = model(target_tensor)
loss = criterion(output, context_tensor)
loss.backward()
optimizer.step()
total_loss += loss.item()
if (epoch + 1) % 10 == 0:
print(f'訓練週期 {epoch+1}, 損失值: {total_loss:.4f}')
# 取得訓練後的詞向量
word_embeddings = model.embeddings.weight.data.numpy()
# 顯示相似詞結果
print("與 'happy' 最相似的詞:", find_similar_words("happy", word_embeddings, word_to_ix, ix_to_word))
print("與 'learning' 最相似的詞:", find_similar_words("learning", word_embeddings, word_to_ix, ix_to_word))
# 視覺化詞向量
pca = PCA(n_components=2)
reduced_embeddings = pca.fit_transform(word_embeddings)
plt.figure(figsize=(8, 6))
for i, word in enumerate(word_to_ix.keys()):
x, y = reduced_embeddings[i]
plt.scatter(x, y)
plt.text(x + 0.01, y + 0.01, word, fontsize=12)
plt.xlabel("PCA 第一維度")
plt.ylabel("PCA 第二維度")
plt.title("Word2Vec Skip-gram 詞向量視覺化")
plt.show()
if __name__ == "__main__":
main()

按 Ctrl + s 儲存

執行程式

& c:/Users/bigred/llm0207/.venv/Scripts/python.exe c:/Users/bigred/llm0207/01_word2vec.py

執行結果 :

訓練週期 10, 損失值: 113.0977
訓練週期 20, 損失值: 101.9441
訓練週期 30, 損失值: 98.7468
訓練週期 40, 損失值: 97.5221
訓練週期 50, 損失值: 96.9120
訓練週期 60, 損失值: 96.5535
訓練週期 70, 損失值: 96.3194
訓練週期 20, 損失值: 101.9441
訓練週期 30, 損失值: 98.7468
訓練週期 40, 損失值: 97.5221
訓練週期 50, 損失值: 96.9120
訓練週期 60, 損失值: 96.5535
訓練週期 70, 損失值: 96.3194
訓練週期 50, 損失值: 96.9120
訓練週期 60, 損失值: 96.5535
訓練週期 70, 損失值: 96.3194
訓練週期 80, 損失值: 96.1548
訓練週期 90, 損失值: 96.0326
訓練週期 100, 損失值: 95.9380
訓練週期 110, 損失值: 95.8620
訓練週期 120, 損失值: 95.7993
訓練週期 130, 損失值: 95.7463
訓練週期 140, 損失值: 95.7004
訓練週期 150, 損失值: 95.6601
訓練週期 160, 損失值: 95.6241
訓練週期 170, 損失值: 95.5915
訓練週期 110, 損失值: 95.8620
訓練週期 120, 損失值: 95.7993
訓練週期 130, 損失值: 95.7463
訓練週期 140, 損失值: 95.7004
訓練週期 150, 損失值: 95.6601
訓練週期 160, 損失值: 95.6241
訓練週期 170, 損失值: 95.5915
訓練週期 160, 損失值: 95.6241
訓練週期 170, 損失值: 95.5915
訓練週期 180, 損失值: 95.5618
訓練週期 190, 損失值: 95.5345
訓練週期 200, 損失值: 95.5090
與 'happy' 最相似的詞: ['learning', 'excited', 'to']
與 'learning' 最相似的詞: ['happy', 'because', 'excited']

然後會 show 出一張圖

範例 2

& c:/Users/bigred/llm0207/.venv/Scripts/python.exe c:/Users/bigred/llm0207/02_stochastic_gradient_descent.py

執行結果

訓練週期 10/100, 損失值: 0.1400
訓練週期 20/100, 損失值: 0.0093
訓練週期 30/100, 損失值: 0.0078
訓練週期 40/100, 損失值: 0.0078
訓練週期 50/100, 損失值: 0.0078
訓練週期 60/100, 損失值: 0.0078
訓練週期 70/100, 損失值: 0.0078
訓練週期 80/100, 損失值: 0.0078
訓練週期 90/100, 損失值: 0.0078
訓練週期 100/100, 損失值: 0.0078
訓練完成！最終模型參數：
參數 linear.weight: 3.0012
參數 linear.bias: 2.0036

範例 3

& c:/Users/bigred/llm0207/.venv/Scripts/python.exe c:/Users/bigred/llm0207/03_vanishing_gradient_problem.py

執行結果

練週期 10/100, 損失值: 0.6906, 第一層梯度: 0.000054
訓練週期 20/100, 損失值: 0.6904, 第一層梯度: 0.000054
訓練週期 30/100, 損失值: 0.6902, 第一層梯度: 0.000055
訓練週期 40/100, 損失值: 0.6899, 第一層梯度: 0.000056
訓練週期 50/100, 損失值: 0.6897, 第一層梯度: 0.000057
訓練週期 60/100, 損失值: 0.6894, 第一層梯度: 0.000058
訓練週期 70/100, 損失值: 0.6892, 第一層梯度: 0.000059
訓練週期 80/100, 損失值: 0.6889, 第一層梯度: 0.000061
訓練週期 90/100, 損失值: 0.6887, 第一層梯度: 0.000062
訓練週期 100/100, 損失值: 0.6884, 第一層梯度: 0.000064
訓練完成！
觀察結果：由於使用 Sigmoid 激活函數，
深層網路中的梯度在反向傳播過程中逐漸消失，
這會導致較早層的權重更新緩慢，影響模型的學習效果。

範例 4

& c:/Users/bigred/llm0207/.venv/Scripts/python.exe c:/Users/bigred/llm0207/04_vanishing_gradient_problem_fixed.py

執行結果

訓練週期 10/100, 損失值: 0.0300, 第一層梯度: 0.002530
訓練週期 20/100, 損失值: 0.0108, 第一層梯度: 0.002441
訓練週期 30/100, 損失值: 0.0057, 第一層梯度: 0.002390
訓練週期 40/100, 損失值: 0.0026, 第一層梯度: 0.000645
訓練週期 50/100, 損失值: 0.0015, 第一層梯度: 0.000145
訓練週期 60/100, 損失值: 0.0010, 第一層梯度: 0.000119
訓練週期 70/100, 損失值: 0.0007, 第一層梯度: 0.000088
訓練週期 80/100, 損失值: 0.0005, 第一層梯度: 0.000051
訓練週期 90/100, 損失值: 0.0004, 第一層梯度: 0.000034
訓練週期 100/100, 損失值: 0.0003, 第一層梯度: 0.000029
測試結果：
輸入: [[ 0.5 0.5]
[-0.5 -0.5]]
預測: [[9.9999988e-01]
[1.6248593e-07]]
改進總結：
1. ReLU 激活函數避免了梯度消失
2. 批次正規化幫助穩定訓練過程
3. Xavier 初始化提供了更好的起始權重
4. Adam 優化器提供了自適應學習率

範例 5

& c:/Users/bigred/llm0207/.venv/Scripts/python.exe c:/Users/bigred/llm0207/05_lstm.py

執行結果

開始序列處理，顯示各時間步的計算結果：
=== 時間步 1 ===
輸入 x: [[-1.266819 -0.23635459 -1.2682884 1.1461157 ]]
遺忘門 (Forget gate) f_t:
[[0.6572256 0.36749864 0.42991826]]
輸入門 (Input gate) i_t:
[[0.59915054 0.5485195 0.48892254]]
候選記憶單元 (Candidate cell) c_candidate:
[[-0.240163 -0.3913692 -0.25401673]]
更新後的記憶單元 (Memory cell) c:
[[-0.1438938 -0.21467364 -0.1241945 ]]
輸出門 (Output gate) o_t:
[[0.5153311 0.620698 0.5811946]]
更新後的隱藏狀態 (Hidden state) h:
[[-0.07364536 -0.13123764 -0.07181233]]
-------------------------------
=== 時間步 2 ===
輸入 x: [[-1.5957438 0.4445498 -0.543539 -1.4150689]]
遺忘門 (Forget gate) f_t:
[[0.41003838 0.46972215 0.5150182 ]]
輸入門 (Input gate) i_t:
[[0.64580995 0.36331752 0.47042245]]
候選記憶單元 (Candidate cell) c_candidate:
[[-0.5844555 0.17626989 -0.62574565]]
更新後的記憶單元 (Memory cell) c:
[[-0.43644914 -0.03679502 -0.35832724]]
輸出門 (Output gate) o_t:
[[0.47196326 0.3124301 0.48176473]]
更新後的隱藏狀態 (Hidden state) h:
[[-0.19383381 -0.01149069 -0.1656017 ]]
-------------------------------
=== 時間步 3 ===
輸入 x: [[ 0.05293205 -0.77091706 -0.7147509 0.17642277]]
遺忘門 (Forget gate) f_t:
[[0.57225716 0.4478708 0.4592267 ]]
輸入門 (Input gate) i_t:
[[0.5396667 0.5573089 0.4244663]]
候選記憶單元 (Candidate cell) c_candidate:
[[-0.43210194 -0.06478254 -0.06248397]]
更新後的記憶單元 (Memory cell) c:
[[-0.48295218 -0.0525833 -0.19107577]]
輸出門 (Output gate) o_t:
[[0.4559599 0.6255029 0.47814628]]
更新後的隱藏狀態 (Hidden state) h:
[[-0.2045458 -0.03286073 -0.0902663 ]]
-------------------------------
=== 時間步 4 ===
輸入 x: [[-0.1387624 -0.6167944 1.6499552 -1.0031953]]
遺忘門 (Forget gate) f_t:
[[0.50970256 0.6360289 0.41048098]]
輸入門 (Input gate) i_t:
[[0.62627995 0.5196672 0.30097124]]
候選記憶單元 (Candidate cell) c_candidate:
[[-0.13491811 -0.45510677 -0.3899505 ]]
更新後的記憶單元 (Memory cell) c:
[[-0.33065847 -0.26994857 -0.19579686]]
輸出門 (Output gate) o_t:
[[0.45622295 0.3666553 0.6378727 ]]
更新後的隱藏狀態 (Hidden state) h:
[[-0.14558636 -0.0966419 -0.12332159]]
-------------------------------
=== 時間步 5 ===
輸入 x: [[ 0.26633266 -0.82728124 -0.04644813 -0.32036436]]
遺忘門 (Forget gate) f_t:
[[0.54177433 0.52325344 0.4516588 ]]
輸入門 (Input gate) i_t:
[[0.5455653 0.55652875 0.38952616]]
候選記憶單元 (Candidate cell) c_candidate:
[[-0.36856094 -0.08281524 -0.14614902]]
更新後的記憶單元 (Memory cell) c:
[[-0.38021633 -0.18734059 -0.14536224]]
輸出門 (Output gate) o_t:
[[0.43234622 0.56785136 0.50058883]]
更新後的隱藏狀態 (Hidden state) h:
[[-0.15689641 -0.1051543 -0.07225849]]
-------------------------------
LSTM 運作機制說明：
1. 遺忘門決定要遺忘多少舊的記憶
2. 輸入門決定要記住多少新的資訊
3. 候選記憶單元產生新的資訊
4. 記憶單元結合舊記憶和新資訊
5. 輸出門控制要輸出的資訊量

範例 6

& c:/Users/bigred/llm0207/.venv/Scripts/python.exe c:/Users/bigred/llm0207/06_tensor_explain.py

執行結果

PyTorch 張量維度展示：
純量張量 : 42
形狀 : torch.Size([])
維度數 : 0
資料型態 : torch.int64
--------------------------------------------------
向量張量 : [1 2 3 4 5]
形狀 : torch.Size([5])
維度數 : 1
資料型態 : torch.int64
--------------------------------------------------
矩陣張量 : [[1 2 3]
[4 5 6]
[7 8 9]]
形狀 : torch.Size([3, 3])
維度數 : 2
資料型態 : torch.int64
--------------------------------------------------
三維張量 : [[[ 1 2 3]
[ 4 5 6]
[ 7 8 9]]
[[10 11 12]
[13 14 15]
[16 17 18]]]
形狀 : torch.Size([2, 3, 3])
維度數 : 3
資料型態 : torch.int64
--------------------------------------------------
四維張量 : [[[[ 1 2]
[ 3 4]]
[[ 5 6]
[ 7 8]]]
[[[ 9 10]
[11 12]]
[[13 14]
[15 16]]]]
形狀 : torch.Size([2, 2, 2, 2])
維度數 : 4
資料型態 : torch.int64
--------------------------------------------------
張量維度的實際應用說明：
0D 張量（純量）：
- 單一數值，如損失函數的輸出值
- 準確率計算的結果
1D 張量（向量）：
- 一維特徵向量
- 單字的詞嵌入表示
2D 張量（矩陣）：
- 批次的特徵向量
- 灰階圖片
- 全連接層的權重矩陣
3D 張量：
- RGB 彩色圖片 (高度 × 寬度 × 色彩通道)
- 時間序列資料 (序列長度 × 批次大小 × 特徵維度)
4D 張量：
- 影像批次 (批次大小 × 高度 × 寬度 × 色彩通道)
- 影片資料 (幀數 × 高度 × 寬度 × 色彩通道)
注意事項：
1. 張量的維度數決定了它可以表示的資料複雜度
2. 在深度學習中，正確理解張量維度對於模型設計至關重要
3. PyTorch 支援自動微分，使得張量運算更加方便

範例 7 : 實作 LSTM 模型來處理給定的文本

& c:/Users/bigred/llm0207/.venv/Scripts/python.exe c:/Users/bigred/llm0207/07_lstm_generate.py

執行結果

開始訓練模型...
訓練週期 10/100, 損失值: 2.8676
訓練週期 20/100, 損失值: 2.8103
訓練週期 30/100, 損失值: 2.7176
訓練週期 40/100, 損失值: 2.0817
訓練週期 50/100, 損失值: 1.1766
訓練週期 60/100, 損失值: 0.5819
訓練週期 70/100, 損失值: 0.2223
訓練週期 80/100, 損失值: 0.1094
訓練週期 90/100, 損失值: 0.0700
訓練週期 100/100, 損失值: 0.0548
使用種子文本 'Hello' 生成的文本：
Hello, this is a sample text for LSTM text for LSTM text for LSTM text for LSTM text for LSTM text for LS

範例 7

& c:/Users/bigred/llm0207/.venv/Scripts/python.exe c:/Users/bigred/llm0207/04_vanishing_gradient_problem_fixed.py

執行結果

訓練週期 10/100, 損失值: 0.0300, 第一層梯度: 0.002530
訓練週期 20/100, 損失值: 0.0108, 第一層梯度: 0.002441
訓練週期 30/100, 損失值: 0.0057, 第一層梯度: 0.002390
訓練週期 40/100, 損失值: 0.0026, 第一層梯度: 0.000645
訓練週期 50/100, 損失值: 0.0015, 第一層梯度: 0.000145
訓練週期 60/100, 損失值: 0.0010, 第一層梯度: 0.000119
訓練週期 70/100, 損失值: 0.0007, 第一層梯度: 0.000088
訓練週期 80/100, 損失值: 0.0005, 第一層梯度: 0.000051
訓練週期 90/100, 損失值: 0.0004, 第一層梯度: 0.000034
訓練週期 100/100, 損失值: 0.0003, 第一層梯度: 0.000029
測試結果：
輸入: [[ 0.5 0.5]
[-0.5 -0.5]]
預測: [[9.9999988e-01]
[1.6248593e-07]]
改進總結：
1. ReLU 激活函數避免了梯度消失
2. 批次正規化幫助穩定訓練過程
3. Xavier 初始化提供了更好的起始權重
4. Adam 優化器提供了自適應學習率

範例 8

& c:/Users/bigred/llm0207/.venv/Scripts/python.exe c:/Users/bigred/llm0207/04_vanishing_gradient_problem_fixed.py

執行結果

訓練週期 10/100, 損失值: 0.0300, 第一層梯度: 0.002530
訓練週期 20/100, 損失值: 0.0108, 第一層梯度: 0.002441
訓練週期 30/100, 損失值: 0.0057, 第一層梯度: 0.002390
訓練週期 40/100, 損失值: 0.0026, 第一層梯度: 0.000645
訓練週期 50/100, 損失值: 0.0015, 第一層梯度: 0.000145
訓練週期 60/100, 損失值: 0.0010, 第一層梯度: 0.000119
訓練週期 70/100, 損失值: 0.0007, 第一層梯度: 0.000088
訓練週期 80/100, 損失值: 0.0005, 第一層梯度: 0.000051
訓練週期 90/100, 損失值: 0.0004, 第一層梯度: 0.000034
訓練週期 100/100, 損失值: 0.0003, 第一層梯度: 0.000029
測試結果：
輸入: [[ 0.5 0.5]
[-0.5 -0.5]]
預測: [[9.9999988e-01]
[1.6248593e-07]]
改進總結：
1. ReLU 激活函數避免了梯度消失
2. 批次正規化幫助穩定訓練過程
3. Xavier 初始化提供了更好的起始權重
4. Adam 優化器提供了自適應學習率

範例 9

& c:/Users/bigred/llm0207/.venv/Scripts/python.exe c:/Users/bigred/llm0207/04_vanishing_gradient_problem_fixed.py

執行結果

訓練週期 10/100, 損失值: 0.0300, 第一層梯度: 0.002530
訓練週期 20/100, 損失值: 0.0108, 第一層梯度: 0.002441
訓練週期 30/100, 損失值: 0.0057, 第一層梯度: 0.002390
訓練週期 40/100, 損失值: 0.0026, 第一層梯度: 0.000645
訓練週期 50/100, 損失值: 0.0015, 第一層梯度: 0.000145
訓練週期 60/100, 損失值: 0.0010, 第一層梯度: 0.000119
訓練週期 70/100, 損失值: 0.0007, 第一層梯度: 0.000088
訓練週期 80/100, 損失值: 0.0005, 第一層梯度: 0.000051
訓練週期 90/100, 損失值: 0.0004, 第一層梯度: 0.000034
訓練週期 100/100, 損失值: 0.0003, 第一層梯度: 0.000029
測試結果：
輸入: [[ 0.5 0.5]
[-0.5 -0.5]]
預測: [[9.9999988e-01]
[1.6248593e-07]]
改進總結：
1. ReLU 激活函數避免了梯度消失
2. 批次正規化幫助穩定訓練過程
3. Xavier 初始化提供了更好的起始權重
4. Adam 優化器提供了自適應學習率

範例 10

& c:/Users/bigred/llm0207/.venv/Scripts/python.exe c:/Users/bigred/llm0207/04_vanishing_gradient_problem_fixed.py

執行結果

訓練週期 10/100, 損失值: 0.0300, 第一層梯度: 0.002530
訓練週期 20/100, 損失值: 0.0108, 第一層梯度: 0.002441
訓練週期 30/100, 損失值: 0.0057, 第一層梯度: 0.002390
訓練週期 40/100, 損失值: 0.0026, 第一層梯度: 0.000645
訓練週期 50/100, 損失值: 0.0015, 第一層梯度: 0.000145
訓練週期 60/100, 損失值: 0.0010, 第一層梯度: 0.000119
訓練週期 70/100, 損失值: 0.0007, 第一層梯度: 0.000088
訓練週期 80/100, 損失值: 0.0005, 第一層梯度: 0.000051
訓練週期 90/100, 損失值: 0.0004, 第一層梯度: 0.000034
訓練週期 100/100, 損失值: 0.0003, 第一層梯度: 0.000029
測試結果：
輸入: [[ 0.5 0.5]
[-0.5 -0.5]]
預測: [[9.9999988e-01]
[1.6248593e-07]]
改進總結：
1. ReLU 激活函數避免了梯度消失
2. 批次正規化幫助穩定訓練過程
3. Xavier 初始化提供了更好的起始權重
4. Adam 優化器提供了自適應學習率

Build HA K8S - 3M

Mon, 06 Feb 2023 13:07:58 +0800

1. 安裝 kubectl , kubeadm , kubelet

安裝 Kubernetes 要用的一些管理命令

$ sudo apk update; sudo apk add kubeadm kubelet kubectl --update-cache --repository http://dl-3.alpinelinux.org/alpine/edge/testing/ --allow-untrusted
$ ssh m2 'sudo apk update; sudo apk add kubeadm kubelet kubectl --update-cache --repository http://dl-3.alpinelinux.org/alpine/edge/testing/ --allow-untrusted'
$ ssh m3 'sudo apk update; sudo apk add kubeadm kubelet kubectl --update-cache --repository http://dl-3.alpinelinux.org/alpine/edge/testing/ --allow-untrusted'

將 m1, m2, m3 的 kubelet 設定為系統自動啟動

$ sudo rc-update add kubelet default; ssh m2 'sudo rc-update add kubelet default'; ssh m3 'sudo rc-update add kubelet default'

檢查有沒有安裝 $ sudo rc-service kubelet status *status: stopped (k8s還沒起來)

$ ssh m2 'sudo rc-service kubelet status';ssh m3 'sudo rc-service kubelet status'

2. kube-vip (set up m1 node)

開機一定會自動產生的 pod
static pods 的 yaml 檔案一定會放在　/etc/kubenetes/manifests
一定會在 node 上面 run

# Set configuration details
#IP 要改成叢集外的新 IP
$ export VIP=120.96.143.xxx (新的IP， 不是 cluster 的 IP)
($ export INTERFACE=eth0 這行可以不用做因為 環境變數 $env 有宣告 GWIF=eth0)
# Configure to use a container runtime
# 取得版本代號
$ export KVVERSION=$(curl -sL https://api.github.com/repos/kube-vip/kube-vip/releases | jq -r ".[0].name")
#檢查
$ echo $KVVERSION
v0.5.7
# podman
$ alias kube-vip="sudo podman run --network host --rm ghcr.io/kube-vip/kube-vip:$KVVERSION"
* image=ghcr.io/kube-vip/kube-vip:$KVVERSION
# ARP
# This configuration will create a manifest that starts kube-vip providing controlplane and services management, using leaderElection. When this instance is elected as the leader it will bind the vip to the specified interface, this is also the same for services of type:LoadBalancer.
# 手動建立 static pods 的目錄
$ sudo mkdir -p /etc/kubernetes/manifests/
$ kube-vip manifest pod \
--interface $GWIF \
--vip $VIP \
--controlplane \
--arp \
--leaderElection | sudo tee /etc/kubernetes/manifests/kube-vip.yaml

--vip : vip 是 kube vip 使用的這些指令都會在 container 裡面執行 ,也就是把 kube-vip 做出來 ,且放到 static pods 的目錄區 ,讓他自動啟動使用內建的環境變數 GWIF 代表 eth0 -> 也代表 ip 位置 . 把 yaml 放到 m1 的 static 目錄區

3. 建立 k8s 系統

命令式

將整個 Clusert 重開，後初始化 K8S

$ sudo kubeadm init --service-cidr 10.98.0.0/24 --pod-network-cidr 10.244.0.0/16 --service-dns-domain=k8s.org --apiserver-advertise-address $IP --control-plane-endpoint "120.96.143.xxx:6443" --upload-certs

--control-plane-endpoint "LOAD_BALANCER_DNS:LOAD_BALANCER_PORT"：代入的DNS及PORT為 kube-apiserver的域名，port預設使用 6443。
--upload-certs：自動將 control plane 會用到的 certificates 儲存到名為 kubeadm-certs 的 secret 中。

設定檔方式

編輯 init-config.yaml

$ nano "${HOME}"/init-config.yaml

檔案內容 :

apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 120.96.143.61 # change from Master node IP
bindPort: 6443
nodeRegistration:
criSocket: unix:///run/crio/crio.sock # change from CRI-O Unix Socket
imagePullPolicy: IfNotPresent
name: k2m1 # change from Master node hsotname
taints: []
---
apiServer:
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: topgun # set your clusterName
controllerManager: {}
dns: {}
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: registry.k8s.io
kind: ClusterConfiguration
kubernetesVersion: 1.26.1
networking:
dnsDomain: k8s.org # DNS domain used by Kubernetes Services.
podSubnet: 10.244.0.0/16 # the subnet used by Pods.
serviceSubnet: 10.98.0.0/24 # subnet used by Kubernetes Services.
scheduler: {}
controlPlaneEndpoint: 120.96.143.60:6443

初始化 K8S 命令

sudo kubeadm init --upload-certs --config="${HOME}"/init-config.yaml

4. 將 bigred 設成 K8S 管理者

$ mkdir -p $HOME/.kube; sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config; sudo chown $(id -u):$(id -g) $HOME/.kube/config

5. 設定 K8S Master 可以執行 Pod

$ kubectl taint node m1 node-role.kubernetes.io/control-plane:NoSchedule- node/m1 untainted

6. 安裝 Flannel 網路套件

$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

7. join m2 and m3 control-plane node

$ ssh m2 'sudo kubeadm join 120.96.143.xxx:6443 --token 9t3pt8.j1smywokfw7qg5ez \
--discovery-token-ca-cert-hash sha256:606851b5dca00e0a375da14ba35bb2c91dbef7ce013757d4422fc25620ee9791 \
--control-plane --certificate-key 4a8d586ecb8706b01ec05547835587d5c7ffca2d18ccde8a824559f22b8e8432'
# ssh 到 m2 將 bigred 設成 K8S 管理者
$ ssh m2 'mkdir -p $HOME/.kube; sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config; sudo chown $(id -u):$(id -g) $HOME/.kube/config'
$ ssh m3 'sudo kubeadm join 120.96.143.xxx:6443 --token 9t3pt8.j1smywokfw7qg5ez \
--discovery-token-ca-cert-hash sha256:606851b5dca00e0a375da14ba35bb2c91dbef7ce013757d4422fc25620ee9791 \
--control-plane --certificate-key 4a8d586ecb8706b01ec05547835587d5c7ffca2d18ccde8a824559f22b8e8432'
# ssh 到 m3 將 bigred 設成 K8S 管理者
$ ssh m3 'mkdir -p $HOME/.kube; sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config; sudo chown $(id -u):$(id -g) $HOME/.kube/config'

create control-plane install command

$ certs=$(sudo kubeadm init phase upload-certs --upload-certs | tail -n 1)
$ echo -e "\nPlease run this command to join master node:\n\n\
sudo $(sudo kubeadm token create --print-join-command) --control-plane --certificate-key ${certs}"

注意 ! 這行命令只能在有 kubeVIP 的那台 master node 執行 !

8. set up m2 node

# Set configuration details
$ export VIP=120.96.143.xxx (新的IP， 不是 cluster 的 IP)
# Configure to use a container runtime
$ export KVVERSION=$(curl -sL https://api.github.com/repos/kube-vip/kube-vip/releases | jq -r ".[0].name")
$ echo $KVVERSION
v0.5.7
# podman
$ alias kube-vip="sudo podman run --network host --rm ghcr.io/kube-vip/kube-vip:$KVVERSION"
# ARP
# This configuration will create a manifest that starts kube-vip providing controlplane and services management, using leaderElection. When this instance is elected as the leader it will bind the vip to the specified interface, this is also the same for services of type:LoadBalancer.
$ kube-vip manifest pod \
--interface $GWIF \
--vip $VIP \
--controlplane \
--arp \
--leaderElection | sudo tee /etc/kubernetes/manifests/kube-vip.yaml

設定 K8S Master 可以執行 Pod $ kubectl taint node m2 node-role.kubernetes.io/control-plane:NoSchedule- node/m1 untainted

9. set up m3 node

# Set configuration details
$ export VIP=120.96.143.xxx (新的IP， 不是 cluster 的 IP)
# Configure to use a container runtime
$ export KVVERSION=$(curl -sL https://api.github.com/repos/kube-vip/kube-vip/releases | jq -r ".[0].name")
$ echo $KVVERSION
v0.5.7
# podman
$ alias kube-vip="sudo podman run --network host --rm ghcr.io/kube-vip/kube-vip:$KVVERSION"
# ARP
# This configuration will create a manifest that starts kube-vip providing controlplane and services management, using leaderElection. When this instance is elected as the leader it will bind the vip to the specified interface, this is also the same for services of type:LoadBalancer.
$ kube-vip manifest pod \
--interface $GWIF \
--vip $VIP \
--controlplane \
--arp \
--leaderElection | sudo tee /etc/kubernetes/manifests/kube-vip.yaml

設定 K8S Master 可以執行 Pod $ kubectl taint node m3 node-role.kubernetes.io/control-plane:NoSchedule- node/m1 untainted

安裝好後，將 m1 主機重新開機

$ sudo reboot

10. 檢視 Kubernetes 系統狀態

檢查 node 的狀態

$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
m1 Ready control-plane 18m v1.25.3
m2 Ready control-plane 11m v1.25.3
m3 Ready control-plane 10m v1.25.3

檢查 kube-system 這個 Namespace 裡面 pod 的狀態

$ kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-565d847f94-mkj5s 1/1 Running 1 62m
coredns-565d847f94-wn6cp 1/1 Running 1 62m
etcd-d6m1 1/1 Running 2 62m
etcd-d6m2 1/1 Running 1 59m
etcd-d6m3 1/1 Running 1 58m
kube-apiserver-d6m1 1/1 Running 2 62m
kube-apiserver-d6m2 1/1 Running 2 59m
kube-apiserver-d6m3 1/1 Running 4 (16m ago) 57m
kube-controller-manager-d6m1 1/1 Running 2 62m
kube-controller-manager-d6m2 1/1 Running 1 58m
kube-controller-manager-d6m3 1/1 Running 1 56m
kube-proxy-2986z 1/1 Running 1 59m
kube-proxy-q7xt6 1/1 Running 2 46m
kube-proxy-v9kc2 1/1 Running 1 45m
kube-proxy-vfpwm 1/1 Running 2 62m
kube-proxy-vlswl 1/1 Running 1 58m
kube-scheduler-d6m1 1/1 Running 2 62m
kube-scheduler-d6m2 1/1 Running 1 59m
kube-scheduler-d6m3 1/1 Running 2 58m
kube-vip-d6m1 1/1 Running 3 62m
kube-vip-d6m2 1/1 Running 1 51m
kube-vip-d6m3 1/1 Running 1 52m

開始建置 Kubernetes Worker

w1, w2 安裝 kubelet 及 kubeadm 命令

# ssh 到 w1 主機安裝 kubelet 及 kubeadm 命令
$ ssh w1 'sudo apk add kubeadm kubelet --update-cache --repository http://dl-3.alpinelinux.org/alpine/edge/testing/ --allow-untrusted'
# ssh 到 w2 主機安裝 kubelet 及 kubeadm 命令
$ ssh w2 'sudo apk add kubeadm kubelet --update-cache --repository http://dl-3.alpinelinux.org/alpine/edge/testing/ --allow-untrusted'
# ssh 到 w1, w2 主機，將 kubelet 設成 Deamon
$ ssh w1 'sudo rc-update add kubelet default'; ssh w2 'sudo rc-update add kubelet default'

w1, w2 加入 K8S Cluster

在 m1 的終端機, 執行以下命令
$ export JOIN=$(echo " sudo `kubeadm token create --print-join-command 2>/dev/null`")
$ ssh w1 "$JOIN"; ssh w2 "$JOIN"
$ ssh w1 sudo reboot; ssh w2 sudo reboot

設定 K8S Worker 標籤

$ kubectl label node w1 node-role.kubernetes.io/worker=; kubectl label node w2 node-role.kubernetes.io/worker=

檢查是否符合預期

$ kubectl get nodes

NAME STATUS ROLES AGE VERSION
m1 Ready control-plane 61m v1.25.0
m2 Ready control-plane 58m v1.25.0
m3 Ready control-plane 57m v1.25.0
w1 Ready worker 45m v1.25.0
w2 Ready worker 44m v1.25.0

測試

$ ks

NAME READY STATUS RESTARTS AGE
coredns-565d847f94-mkj5s 1/1 Running 1 3h5m
coredns-565d847f94-wn6cp 1/1 Running 1 3h5m
etcd-d6m1 1/1 Running 2 3h5m
etcd-d6m2 1/1 Running 1 3h2m
etcd-d6m3 1/1 Running 1 3h
kube-apiserver-d6m1 1/1 Running 2 3h5m
kube-apiserver-d6m2 1/1 Running 2 3h2m
kube-apiserver-d6m3 1/1 Running 4 (139m ago) 3h
kube-controller-manager-d6m1 1/1 Running 2 3h5m
kube-controller-manager-d6m2 1/1 Running 1 3h1m
kube-controller-manager-d6m3 1/1 Running 1 179m
kube-proxy-2986z 1/1 Running 1 3h2m
kube-proxy-q7xt6 1/1 Running 2 169m
kube-proxy-v9kc2 1/1 Running 1 168m
kube-proxy-vfpwm 1/1 Running 2 3h5m
kube-proxy-vlswl 1/1 Running 1 3h1m
kube-scheduler-d6m1 1/1 Running 2 3h5m
kube-scheduler-d6m2 1/1 Running 1 3h2m
kube-scheduler-d6m3 1/1 Running 2 3h1m
kube-vip-d6m1 1/1 Running 3 3h5m
kube-vip-d6m2 1/1 Running 1 174m
kube-vip-d6m3 1/1 Running 1 175m

$ kubectl logs -n kube-system kube-vip-d6m1

time="2022-12-20T08:16:19Z" level=info msg="Starting kube-vip.io [v0.5.7]"
time="2022-12-20T08:16:19Z" level=info msg="namespace [kube-system], Mode: [ARP], Features(s): Control Plane:[true], Services:[false]"
time="2022-12-20T08:16:19Z" level=info msg="prometheus HTTP server started"
time="2022-12-20T08:16:19Z" level=info msg="Starting Kube-vip Manager with the ARP engine"
time="2022-12-20T08:16:19Z" level=info msg="Beginning cluster membership, namespace [kube-system], lock name [plndr-cp-lock], id [d6m1]"
I1220 08:16:19.293587 1 leaderelection.go:248] attempting to acquire leader lease kube-system/plndr-cp-lock...
time="2022-12-20T08:16:20Z" level=info msg="Node [d6m2] is assuming leadership of the cluster"
time="2022-12-20T08:16:26Z" level=info msg="Node [d6m3] is assuming leadership of the cluster"

$ kubectl logs -n kube-system kube-vip-d6m2

time="2022-12-20T08:19:20Z" level=info msg="Starting kube-vip.io [v0.5.7]"
time="2022-12-20T08:19:20Z" level=info msg="namespace [kube-system], Mode: [ARP], Features(s): Control Plane:[true], Services:[false]"
time="2022-12-20T08:19:20Z" level=info msg="prometheus HTTP server started"
time="2022-12-20T08:19:20Z" level=info msg="Starting Kube-vip Manager with the ARP engine"
time="2022-12-20T08:19:20Z" level=info msg="Beginning cluster membership, namespace [kube-system], lock name [plndr-cp-lock], id [d6m2]"
I1220 08:19:20.510887 1 leaderelection.go:248] attempting to acquire leader lease kube-system/plndr-cp-lock...
time="2022-12-20T08:19:20Z" level=info msg="Node [d6m3] is assuming leadership of the cluster"

$ kubectl logs -n kube-system kube-vip-d6m3

............
E1220 08:16:01.515714 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-cp-lock: Get "https://kubernetes:6443/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock": dial tcp 127.0.0.1:6443: connect: connection refused
E1220 08:16:13.254844 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-cp-lock: Get "https://kubernetes:6443/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock": net/http: TLS handshake timeout
time="2022-12-20T08:16:20Z" level=info msg="Node [d6m2] is assuming leadership of the cluster"
I1220 08:16:25.734487 1 leaderelection.go:258] successfully acquired lease kube-system/plndr-cp-lock
time="2022-12-20T08:16:25Z" level=info msg="Node [d6m3] is assuming leadership of the cluster"
time="2022-12-20T08:16:25Z" level=info msg="Gratuitous Arp broadcast will repeat every 3 seconds for [120.96.143.248]"

bigred@d6m1:~~$ sudo poweroff bigred@d6m2:~~$ kg all

NAME READY STATUS RESTARTS AGE
pod/presenation-bd9bb89cd-ptszx 1/1 Running 0 13s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.98.0.1 <none> 443/TCP 3h9m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/presenation 1/1 1 1 14s
NAME DESIRED CURRENT READY AGE
replicaset.apps/presenation-bd9bb89cd 1 1 1 14s

一開始 m1 當 leadership of the cluster ，當我們把 m1 關機

pod 還是正常 running
leadership of the cluster 變成 m3

$ kubectl logs -n kube-system kube-vip-d6m2

time="2022-12-20T08:19:20Z" level=info msg="Starting kube-vip.io [v0.5.7]"
time="2022-12-20T08:19:20Z" level=info msg="namespace [kube-system], Mode: [ARP], Features(s): Control Plane:[true], Services:[false]"
time="2022-12-20T08:19:20Z" level=info msg="prometheus HTTP server started"
time="2022-12-20T08:19:20Z" level=info msg="Starting Kube-vip Manager with the ARP engine"
time="2022-12-20T08:19:20Z" level=info msg="Beginning cluster membership, namespace [kube-system], lock name [plndr-cp-lock], id [d6m2]"
I1220 08:19:20.510887 1 leaderelection.go:248] attempting to acquire leader lease kube-system/plndr-cp-lock...
time="2022-12-20T08:19:20Z" level=info msg="Node [d6m3] is assuming leadership of the cluster"

$ kubectl logs -n kube-system kube-vip-d6m3

E1220 08:16:13.254844 1 leaderelection.go:330] error retrieving resource lock kube-system/plndr-cp-lock: Get "https://kubernetes:6443/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/plndr-cp-lock": net/http: TLS handshake timeout
time="2022-12-20T08:16:20Z" level=info msg="Node [d6m2] is assuming leadership of the cluster"
I1220 08:16:25.734487 1 leaderelection.go:258] successfully acquired lease kube-system/plndr-cp-lock
time="2022-12-20T08:16:25Z" level=info msg="Node [d6m3] is assuming leadership of the cluster"
time="2022-12-20T08:16:25Z" level=info msg="Gratuitous Arp broadcast will repeat every 3 seconds for [120.96.143.248]"

重新建立損壞 master

參考文件

在 Alpine Linux 安裝 etcd-ctl 命令套件檔

$ sudo apk add etcd-ctl \
--no-cache \
--update-cache \
--allow-untrusted \
--repository http://dl-cdn.alpinelinux.org/alpine/edge/testing

$ kubectl get pods -n kube-system | grep etcd

etcd-d4m1 1/1 Running 8 8d
etcd-d4m2 1/1 Running 8 8d
etcd-d4m3 0/1 CrashLoopBackOff 1020 (3m ago) 19h

清除損壞的 master node

$ ks

NAME READY STATUS RESTARTS AGE
coredns-565d847f94-h6kwm 1/1 Running 7 15d
coredns-565d847f94-rjmkc 1/1 Running 7 15d
etcd-d3m1 1/1 Running 8 15d
etcd-d3m2 0/1 CrashLoopBackOff 885 (2m1s ago) 9h
etcd-d3m3 1/1 Running 8 15d
kube-apiserver-d3m1 1/1 Running 22 15d
kube-apiserver-d3m2 0/1 CrashLoopBackOff 866 (4m15s ago) 6d21h
kube-apiserver-d3m3 1/1 Running 23 (9h ago) 15d
kube-controller-manager-d3m1 1/1 Running 10 15d
kube-controller-manager-d3m2 1/1 Running 12 6d21h
kube-controller-manager-d3m3 1/1 Running 12 (9h ago) 15d
kube-proxy-8m84p 1/1 Running 7 15d
kube-proxy-9hwvx 1/1 Running 9 15d
kube-proxy-hvmvl 1/1 Running 9 15d
kube-proxy-j4l4m 1/1 Running 0 6d21h
kube-proxy-zn9d2 1/1 Running 8 15d
kube-scheduler-d3m1 1/1 Running 10 15d
kube-scheduler-d3m2 1/1 Running 11 6d21h
kube-scheduler-d3m3 1/1 Running 13 (9h ago) 15d
kube-vip-d3m1 1/1 Running 24 15d
kube-vip-d3m2 1/1 Running 11 6d21h
kube-vip-d3m3 1/1 Running 28 (9h ago) 15d

$ kubectl get nodes
$ kubectl drain <node-name>
$ kubectl drain <node-name> --ignore-daemonsets --delete-local-data
$ kubectl delete node <node-name>

找到那一台壞掉 master node 的 ID

Find the id of the member to remove

sudo ETCDCTL_API=3 etcdctl --endpoints 127.0.0.1:2379 --cacert /etc/kubernetes/pki/etcd/ca.crt --cert /etc/kubernetes/pki/etcd/server.crt --key /etc/kubernetes/pki/etcd/server.key member list

出現以下的結果

78c846a8a9465434, started, d3m3, https://120.96.143.183:2380, https://120.96.143.183:2379, false
92df15cb2da41804, started, d3m1, https://120.96.143.181:2380, https://120.96.143.181:2379, false
d3aa799c592c5386, started, d3m2, https://120.96.143.182:2380, https://120.96.143.182:2379, false

把損壞刪除 ==I want to remove d3aa799c592c5386==

sudo ETCDCTL_API=3 etcdctl --endpoints 127.0.0.1:2379 --cacert /etc/kubernetes/pki/etcd/ca.crt --cert /etc/kubernetes/pki/etcd/server.crt --key /etc/kubernetes/pki/etcd/server.key member remove d3aa799c592c5386

Checking, 檢查

sudo ETCDCTL_API=3 etcdctl --endpoints 127.0.0.1:2379 --cacert /etc/kubernetes/pki/etcd/ca.crt --cert /etc/kubernetes/pki/etcd/server.crt --key /etc/kubernetes/pki/etcd/server.key member list

Now I can join my new master.

Join a Node to a Kubernetes Cluster

$ kubectl get cm kubeadm-config -n kube-system -o yaml

apiVersion: v1
data:
ClusterConfiguration: |
apiServer:
extraArgs:
authorization-mode: Node,RBAC
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: 120.96.143.194:6443
controllerManager: {}
dns: {}
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: registry.k8s.io
kind: ClusterConfiguration
kubernetesVersion: v1.25.5
networking:
dnsDomain: k8s.org
podSubnet: 10.244.0.0/16
serviceSubnet: 10.98.0.0/24
scheduler: {}
kind: ConfigMap
metadata:
creationTimestamp: "2022-12-20T07:12:01Z"
name: kubeadm-config
namespace: kube-system
resourceVersion: "199"
uid: 75169c8c-9389-4f20-bacd-56b3f182f64d

複製以上那段貼到新的 nano 檔案

Copy the marked text under ClusterConfiguration:, create a new file to store the configuration

$ nano kubeadm-config.yaml

 apiServer:
extraArgs:
authorization-mode: Node,RBAC
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: 120.96.143.194:6443
controllerManager: {}
dns: {}
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: registry.k8s.io
kind: ClusterConfiguration
kubernetesVersion: v1.25.5
networking:
dnsDomain: k8s.org
podSubnet: 10.244.0.0/16
serviceSubnet: 10.98.0.0/24
scheduler: {}

Get Kubeadm Join Command for Master Node

$ sudo kubeadm init phase upload-certs --upload-certs --config kubeadm-config.yaml

[upload-certs] Storing the certificates in Secret “kubeadm-certs” in the “kube-system” Namespace [upload-certs] Using certificate key: ==ec52acebb6630efa584b6ff784db4a50ac59f8c22bc81baab6b82c1e70244787==

The marked part above is the ==certificate key== that we need to pass to the join command. 上面標記的部分是我們需要傳遞給加入命令的證書密鑰。

$ kubeadm token create --print-join-command

==kubeadm join 120.96.143.194:6443 –token nlzfjg.6e5x6p2hb36rcjd3 –discovery-token-ca-cert-hash sha256:2a3bd2c4c5a6de8332503d95dcdef141f45790504f5a2c11ea96c06bd86ca87c==

Joining Master Node to Kubernetes CLuster

Now, execute the join command with this format

<basic join command> --control-plane --certificate-key <ceritifcate-key>

In this case, I will execute this command to join my new master node:

==$ ssh <壞掉的 node>==

$ sudo kubeadm join 120.96.143.194:6443 --token nlzfjg.6e5x6p2hb36rcjd3 --discovery-token-ca-cert-hash sha256:2a3bd2c4c5a6de8332503d95dcdef141f45790504f5a2c11ea96c06bd86ca87c --control-plane --certificate-key ec52acebb6630efa584b6ff784db4a50ac59f8c22bc81baab6b82c1e70244787

如果不是全新的機器要把舊的檔案，目錄刪除

$ sudo rm -r /etc/kubernetes/kubelet.conf $ sudo rm -r /etc/kubernetes/manifests/ $ sudo rm -r /var/lib/etcd $ sudo reboot $ sudo rc-service kubelet stop

加到 kube-vip

$ mkdir -p $HOME/.kube $ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

cp: overwrite ‘/home/bigred/.kube/config’? ==y==

$ sudo chown $(id -u):$(id -g) $HOME/.kube/config $ kubectl get nodes

NAME STATUS ROLES AGE VERSION
d4m1 Ready control-plane 9d v1.25.0
d4m2 Ready control-plane 9d v1.25.0
d4m3 Ready control-plane 18m v1.25.0
d4w1 Ready worker 8d v1.25.0
d4w2 Ready worker 8d v1.25.0

#取得版本代號 $ export KVVERSION=$(curl -sL https://api.github.com/repos/kube-vip/kube-vip/releases | jq -r ".[0].name")

#檢查 $ echo $KVVERSION v0.5.7

#podman $ alias kube-vip="sudo podman run --network host --rm ghcr.io/kube-vip/kube-vip:$KVVERSION"

$ sudo mkdir -p /etc/kubernetes/manifests/

$ kube-vip manifest pod \
--interface eth0 \
--vip 120.96.143.194 \
--controlplane \
--arp \
--leaderElection | sudo tee /etc/kubernetes/manifests/kube-vip.yaml

==$ kubectl taint node node-role.kubernetes.io/control-plane:NoSchedule-== node/ untainted

$ ks

NAME READY STATUS RESTARTS AGE
coredns-565d847f94-69lpx 1/1 Running 9 9d
coredns-565d847f94-r626m 1/1 Running 9 9d
etcd-d4m1 1/1 Running 9 9d
etcd-d4m2 1/1 Running 8 9d
etcd-d4m3 1/1 Running 0 12m
kube-apiserver-d4m1 1/1 Running 14 9d
kube-apiserver-d4m2 1/1 Running 13 9d
kube-apiserver-d4m3 1/1 Running 0 12m
kube-controller-manager-d4m1 1/1 Running 13 9d
kube-controller-manager-d4m2 1/1 Running 15 (16m ago) 9d
kube-controller-manager-d4m3 1/1 Running 0 12m
kube-proxy-crwbg 1/1 Running 11 9d
kube-proxy-hbndh 1/1 Running 9 9d
kube-proxy-rnfxc 1/1 Running 8 9d
kube-proxy-t4bgz 1/1 Running 1 26m
kube-proxy-txfn6 1/1 Running 8 9d
kube-scheduler-d4m1 1/1 Running 13 9d
kube-scheduler-d4m2 1/1 Running 16 (16m ago) 9d
kube-scheduler-d4m3 1/1 Running 0 12m
kube-vip-d4m1 1/1 Running 26 9d
kube-vip-d4m2 1/1 Running 26 (21h ago) 9d
kube-vip-d4m3 1/1 Running 0 77s

tags: `K8S-陳老師-第6期`

CKA

Sun, 23 Oct 2022 05:06:04 +0800

參考文件 1.https://zhuanlan.zhihu.com/p/564737349 2.https://github.com/kubernetes-csi/external-snapshotter 3.https://github.com/kubernetes-csi/csi-driver-host-path 4.https://github.com/kubernetes-csi/csi-driver-host-path/blob/master/examples/csi-storageclass.yaml 5.https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/ha-topology/ 6.https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/ha-topology/ 7.https://rafay.co/the-kubernetes-current/etcd-kubernetes-what-you-should-know/

考試環境圖

左上角有一個筆記本可以使用，方便寫 yaml 檔（Applications 的 Mousepad）

1. Scale

題型 1：Scale the deployment presenation to 3 pods 題型 2：Scale the deployment loadbalabcer to 6 pods

環境準備

$ kubectl create deploy presenation --image=quay.io/cloudwalker/nginx
$ kubectl create deploy loadbalabcer --image=quay.io/cloudwalker/nginx
$ kubectl get all

解答

題型 1

$ kubectl scale deploy presentaion –replicas=3

$ kubectl get all

題型 2

$ kubectl scale deploy loadbalabcer –replicas=6

$ kubectl get all

2. NetworkPolicy （7%）

Create new NetworkPolicy named allow-port-from-namespace that allows Pods in the existing namespace internal to connect to port 9000 of other Pods in the same namespace.

Ensure that the new NetworkPolicy:

does not allow access to Pods not listening on port 9000
does not allow access from Pods not in namespace internal

解答

K8S DOC 查詢關鍵字 : networkpolicy
找 Network Policies | Kubernetes 這篇
用第一個 YANL 檔

檢查 namespace，如果沒有這個 namespace 就直接 create $ k get namespace

看 Namespace 的標籤

$ k get ns internal --show-labels
NAME STATUS AGE LABELS
internal Active 3m27s kubernetes.io/metadata.name=internal

編輯 YAML 檔

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-port-from-namespace
namespace: internal
spec:
podSelector: {} # 新增 {}
# matchLabels:
# role: db
policyTypes:
- Ingress
ingress:
- from:
# - ipBlock: # remove
# cidr: 172.17.0.0/16 # remove
# except: # remove
# - 172.17.1.0/24 # remove
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: internal
# - podSelector: # remove
# matchLabels: # remove
# role: frontend # remove
ports:
- protocol: TCP
port: 9000 # port number 改成 9000
# egress: # remove
# - to: # remove
# - ipBlock: # remove
# cidr: 10.0.0.0/24 # remove
# ports: # remove
# - protocol: TCP # remove
# port: 5978 # remove

驗證

此功能在 flannel 無法成功
因此使用 Calico ，它的功能比較多

撰寫 yaml

# 建立 bx pod 並且架網站 port 9000
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: bx
name: bx
namespace: internal
spec:
containers:
- image: quay.io/cloudwalker/busybox
name: bx
command:
- /bin/sh
- -c
- |
mkdir www
echo hi > www/index.html
busybox httpd -p 9000 -h www -f
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}

驗證同樣在 internal namespace 的 pod 可以透過 9000 port 連線

$ kubectl run alp -it --image=quay.io/cloudwalker/alpine -n internal
# nc -w 1 10.244.190.72 9000;echo $?
0

在 internal namespace 以外都無法透過 9000 port 連線

$ kubectl run alp -it --image=quay.io/cloudwalker/alpine
# nc -w 1 10.244.190.72 9000;echo $?
1

2.2 NetworkPolicy

Create a new NetworkPolicy named allow-port-from-namespace in the existing namespace echo. Ensure that the new NetworkPolicy allows Pods in namespace my-app to connect to port 9000 of Pods in namespace echo.

Further ensure that the new NetworkPolicy:

• does not allow access to Pods, which don’t listen on port 9000 • does not allow access from Pods, which are not in namespace my-app

解答

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-port-from-namespace
namespace: echo
spec:
podSelector: {}
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: my-app
ports:
- protocol: TCP
port: 9000

3. Multi-container

題型 1: Create a pod named kucc8 with a single app container for each of the following images runing inside(there may be between 1 and 4 images specified): nginx + memcached

題型 2: Create a pod named kucc1 with a single app container for each of the following images runing inside(there may be between 1 and 4 images specified): nginx + redis + memcached + consul

解答

題型 1:

產生 YAML 檔 $ export do="--dry-run=client -o yaml"

$ kubectl run kucc8 –image=nginx $do > pod.yaml 編輯 YAML 檔

apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: kucc8
name: kucc8
spec:
containers:
- image: nginx
name: kucc8
resources: {}
- image: memcached # add and change
name: kucc8-1 # add
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}

產生 pod

$ kubectl apply -f pod.yaml

題型 2:

產生 YAML 檔

$ kubectl run kucc1 –image=nginx $do > pod2.yaml

編輯 YAML 檔

apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: kucc1
name: kucc1
spec:
containers:
- image: nginx
name: kucc1
resources: {}
- image: redis # add and change
name: kucc1-1 # add
- image: memcached # add and change
name: kucc1-2 # add
- image: consul # add and change
name: kucc1-3 # add
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}

產生 pod

$ kubectl apply -f pod2.yaml

4. CPU

From the pod label name=cpu-loader,find pods running high CPU workloads and write the name of the pod consuming most CPU to the file /opt/KUTR00401/KURT00401.txt (while alreay exists).

環境準備

$ kubectl run nginx --image=quay.io/cloudwalker/nginx --labels=name=cpu-loader
$ kubectl run nginx2 --image=quay.io/cloudwalker/nginx --labels=name=cpu-loader
$ k run alpine1 --image=quay.io/cloudwalker/nginx --labels=name=cpu-loader -- yes > /dev/null

解答

$ kubectl top pod -l “name=cpu-loader” –sort-by=cpu -A

NAME CPU(cores) MEMORY(bytes)
alpine1 9m 0Mi
nginx 0m 6Mi
nginx2 0m 2Mi

$ echo “alpine1” > /opt/KUTR00401/KURT00401.txt

5. NodeSelector

Schedule a pod as follows:

name: nginx-kusc00401
Image: nginx
Node selector: disk=spinning

解答

產生 YAML 檔

$ kubectl get nodes --show-labels | grep "disk=spinning"

$ kubectl label node w1 disk=spinning $ kubectl run nginx-kusc00401 –image=nginx –dry-run=client -o yaml > pod.yaml

編輯 YAML 檔

apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: nginx-kusc00401
name: nginx-kusc00401
spec:
containers:
- image: nginx
name: nginx-kusc00401
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
nodeSelector: # add
disk: spinning # add
status: {}

產生 pod

$ kubectl apply -f pod.yaml 檢查

$ kubectl get pod -o wide

6. Kubernetes 升級

Given an existing Kubernetes cluster running version 1.24.1, upgrade all of Kubernetes control plane and node components on the master node only to version 1.24.2

You are also expected to upgrade kubelet and kubectl on the master node.

Be sure to drain the master node before upgrading it and uncordon it after the upgrade. Do not upgrade the worker nodes, etcd, the container manager, the CNI plugin, the DNS service or any other addons.

考試環境 3台機器:1台 master 2台 worker

drain：進入維護模式，會把這台 nodes 上的 pod 趕到別台 node 上，有人照顧(例如:deployment)的 pod 才會在別台 node 產生，沒人照顧的 pod 就會被刪除。
uncordon：解除維護模式，之後就可以在這台 node 產生 pod，但被趕走的 pod 並不會在回到這台 node 上。

解答文件1 文件2

K8S DOC 查詢關鍵字 : upgrading kubeadm
找 Upgrading kubeadm clusters | Kubernetes 這篇對 Linux 套件 kubeadm 進行升級

# ssh 連進 control plane node
$ ssh m1
# 提權，因為考試的環境使用者沒有 sudo 的權限
$ sudo -i
# 這是 ubuntu 的更新套件清單命令
$ apt update
# 看當前主機可以升級 kubeadm 的那些版本
$ apt-cache madison kubeadm | grep 1.24.2
kubeadm | 1.24.2-00 | https://apt.kubernetes.io kubernetes-xenial/main amd64 Packages
# 將 kubeadm 升級至 1.24.2 版
# unhold ，因為現在要升級，解除鎖定
# hold ，讓系統 update 或 upgrade 的時候，忽略 hold 的套件，以後系統在 update upgrade 的時候就不會升級這些套件(鎖定版本)
# 這裡只是針對套件做升級而已，並沒有升級 kubernetes
# alpine 沒有鎖定版本的功能，只要 upgrade 就全部升級了
$ apt-mark unhold kubeadm && \
apt-get update && apt-get install -y kubeadm=1.24.2-00 && \
apt-mark hold kubeadm
# 檢查下載下來的 kubeadm 版本
$ kubeadm version

# Drain control-plane node
$ kubectl drain m1 --ignore-daemonsets
# 已成功 drain m1
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
m1 Ready,SchedulingDisabled control-plane 51m v1.24.1
w1 Ready <none> 51m v1.24.1
w2 Ready <none> 51m v1.24.1

kubeadm upgrade

#檢查下載下來的 kubeadm 版本 $ kubeadm version

#將 kubeadm 升級至指定版本 : 1.24.2，可以不用打

$ sudo kubeadm upgrade apply v1.24.2 –etcd-upgrade=false

$ kubectl apply -f pod.yaml

成功後，會看到的螢幕輸出

[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.24.2". Enjoy!
[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.

Upgrade kubelet and kubectl

$ apt-mark unhold kubelet kubectl && \
apt-get update && apt-get install -y kubelet=1.24.2-00 kubectl=1.24.2-00 && \
apt-mark hold kubelet kubectl

Restart the kubelet

# Reload systemd manager configuration
$ systemctl daemon-reload
# 將 kubelet 重啟
$ systemctl restart kubelet
# 檢查 kubelet 狀態
$ systemctl status kubelet

回到 bigred
Uncordon the node

# Bring the node back online by marking it schedulable
$ kubectl uncordon m1
檢查 node 狀態

$ kubectl get nodes

7. Service

Reconfigure the existing deployment front-end and add a port specifiction named http exposing port 80/tcp of the existing container nginx.

Create a new service named front-end-svc exposing the container port http.

Configure the new service to also expose the individual Pods via a NodePort on the nodes on which they are scheduled.

解答

參考文章：https://blog.cptsai.com/2020/11/15/k8s-external-traffic-policy/

K8S DOC 查詢關鍵字 : nodeport yaml
找 Connecting Applications with Services | Kubernetes 這篇
service/networking/nginx-secure-app.yaml 這個 YAML 檔

匯出 YAML 檔 $ kubectl get deploy front-end -o yaml > deploy.yaml

編輯 YAML 檔

apiVersion: apps/v1
kind: Deployment
......
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- name: http # add
protocol: TCP # add
containerPort: 80 # add

$ k get pod –show-labels

front-end-6bc5dc8cbb-mv7hb 1/1 Running 0 33m app=front-end,pod-template-hash=6bc5dc8cbb

重新建立

$ kubectl replace -f deploy.yaml –force

編輯 Service YAML 檔

# externalTrafficPolicy: Local 預設是 Cluster，Local 代表這個 node 要有 pod 才能透過 nodeport 連進來
apiVersion: v1
kind: Service
metadata:
name: front-end-svc
# labels: # remove
# run: my-nginx # remove
spec:
type: NodePort
externalTrafficPolicy: Local
# internalTrafficPolicy: Local # 不確定，看文章
ports:
- port: 80
targetPort: 80
protocol: TCP
name: http
# - port: 443 # remove
# protocol: TCP # remove
# name: https # remove
selector:
run: front-end # 看考試時的標籤

產生 Service

$ kubectl apply -f srv.yaml

用命令產生 $ kubectl expose deployment front-end –port=80 –target-port=80 –type=NodePort –name=front-end-svc

8. RBAC

Context You have been asked to create a new ClusterRole for a deployment pipeline and bind it to a specific ServiceAccount scoped a specific namespace.

Task Create a new ClusterRole named deployment-clusterrole that only allows the creation of the following resource types:

Deployment
StatefulSet
DaemonSet Create a new ServiceAccount named cicd-token in the existing namespace app-team1.

Limited to namespace app-team1, bind the new ClusterRole deployment-clusterrole to the new ServiceAccount cicd-token.

解答

建立 clusterrole $ kubectl create clusterrole deployment-clusterrole –verb=create –resource=Deployment,StatefulSet,DaemonSet
建立 ServiceAccount $ kubectl create sa cicd-token -n app-team1
建立 rolebinding $ kubectl create -n app-team1 rolebinding haha
–clusterrole=deployment-clusterrole
–serviceaccount=app-team1:cicd-token
檢查

$ k create deploy pipi -n app-team1 --image=quay.io/cloudwalker/alpine --as system:serviceaccount:app-team1:cicd-token
$ k get all -n app-team1
$ k delete deployment.apps/pipi -n app-team1 --as system:serviceaccount:app-team1:cicd-token
-Error from server (Forbidden): deployments.apps "pipi" is forbidden: User "system:serviceaccount:app-team1:cicd-token" cannot delete resource "deployments" in API group "apps" in the namespace "app-team1"

使用 can-i 檢查

$ k auth can-i --list -n app-team1 --as system:serviceaccount:app-team1:cicd-token

$ k auth can-i create pod -n app-team1 --as system:serviceaccount:app-team1:cicd-token
no
$ k auth can-i create deploy -n app-team1 --as system:serviceaccount:app-team1:cicd-token
yes
$ k auth can-i create sts -n app-team1 --as system:serviceaccount:app-team1:cicd-token
yes
$ k auth can-i create ds -n app-team1 --as system:serviceaccount:app-team1:cicd-token
yes

9. Check Ready Node

Check to see how many nodes are ready (not including nodes tained NoSchedule) and write the number to /opt/KUSC00402/kusc00402.txt.

解答

$ kubectl get nodes | grep 'Ready'

$ kubectl describe node | grep ‘NoSchedule’ $ echo “{數量}” > /opt/KUSC00402/kusc00402.txt

10. PVC

Create a new PersistentVolumeClaim:

Name: pv-volume
Class: csi-hostpath-sc
Capacity: 10Mi

Create a new Pod which mounts the PersistentVolumeClaim as a volume:

Name: web-server
Image: nginx
Mount Path: /usr/share/nginx/html

Configure the new Pod to have ReadWriteOnce access on the volume.

Finally,using kubectl edit or kubectl patch expand the PersistentVolumeClaim to a capacity of 70Mi and ==record that change.==

解答

K8S DOC 查詢關鍵字 : pod pv
找 Configure a Pod to Use a PersistentVolume for Storage | Kubernetes 這篇
找 pods/storage/pv-claim.yaml 這篇 YAML 檔
和 pods/storage/pv-pod.yaml 這篇 YAML 檔編輯 PVC YANL 檔

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pv-volume # change
spec:
storageClassName: csi-hostpath-sc # change
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi # change

產生 PVC

$ kubectl apply -f pvc.yaml 編輯 pod YAML

$ kubectl run web-server –image=nginx –dry-run=client -o yaml > pod.yaml

apiVersion: v1
kind: Pod
metadata:
name: web-server # change
spec:
volumes: # add
- name: task-pv-storage # add
persistentVolumeClaim: # add
claimName: pv-volume # change
containers:
- name: task-pv-container
image: nginx
ports:
- containerPort: 80
name: "http-server"
volumeMounts: # add
- mountPath: "/usr/share/nginx/html" # add
name: task-pv-storage # add

產生 Pod $ kubectl apply -f pod.yaml

將 PVC 的 capacity 調成 70Mi 用 nano 編輯 $ export KUBE_EDITOR=nano

修改 YAML 檔，將 capacity 設為 70Mi kubectl get pvc pv-volume

==記得加 record== $ k edit pvc pv-volume –record

#檢查
$ k get pvc pv-volume

11. Pod Logs

Monitor the logs of pod bar and:

Extract log lines corresponding to error unable-to-access-website
Write them to /opt/KUTR00101/bar

解答

$ kubectl logs bar | grep 'unable-to-access-website' > /opt/KUTR00101/bar

12. PV

題型 1: Create a persistent volume with name app-config, of capacity 1Gi and access mode ReadOnlyMany, the type of volume is hostPath and its location is /svc/app-config.

題型 2： Create a persistent volume with name app-config, of capacity 2Gi and access mode ReadWriteMany, the type of volume is hostPath and its location is /srv/app-config.

題型 1: 解答

K8S DOC 查詢關鍵字 : pod pv 找 Configure a Pod to Use a PersistentVolume for Storage | Kubernetes 這篇找 pods/storage/pv-volume.yaml 這篇 YAML 檔編輯 PV YAML 檔

apiVersion: v1
kind: PersistentVolume
metadata:
name: app-config # change
# labels: # remove
# type: local # remove
spec:
# storageClassName: manual # remove
capacity:
storage: 1Gi # change
accessModes:
- ReadOnlyMany # change
hostPath:
path: "/svc/app-config" # change

產生 PV

$ kubectl apply -f pv.yaml

題型 2: 解答

編輯 PV YAML 檔

apiVersion: v1
kind: PersistentVolume
metadata:
name: app-config # change
# labels: # remove
# type: local # remove
spec:
# storageClassName: manual # remove
capacity:
storage: 2Gi # change
accessModes:
- ReadWriteMany # change
hostPath:
path: "/srv/app-config" # change

產生 PV

$ kubectl apply -f pv.yaml

13. cordon & drain

Set the node named ek8s-node-1 as unavaliable and reschedule all pods running on it.

解答

參考連結: https://kubernetes.io/docs/concepts/workloads/pods/disruptions/

#不可以在安排 pod 在這個 node 產生 $ kubectl cordon ek8s-node-1

$ kubectl drain ek8s-node-1 –ignore-daemonsets –delete-emptydir–data –force #把 pod 趕到別的 node #–ignore-daemonsets 忽略 DaemonSet 管理的 pod #–force 把沒有 controller 管理的 pod 刪除

Applying a Node Cordon
Cordoning a Node marks it as unavailable to the Kubernetes scheduler. The Node will be ineligible to host any new Pods subsequently added to your cluster.
Use the kubectl cordon command to place a cordon around a named Node:
$ kubectl cordon node-1
node/node-1 cordoned
Existing Pods already running on the Node won’t be affected by the cordon. They’ll remain accessible and will still be hosted by the cordoned Node.
You can check which of your Nodes are currently cordoned with the get nodes command:
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
node-1 Ready,SchedulingDisabled control-plane,master 26m v1.23.3
Cordoned nodes appear with the SchedulingDisabled status.
Draining a Node
The next step is to drain remaining Pods out of the Node. This procedure will evict the Pods so they’re rescheduled onto other Nodes in your cluster. Pods are allowed to gracefully terminate before they’re forcefully removed from the target Node.
Run kubectl drain to initiate a drain procedure. Specify the name of the Node you’re taking out for maintenance:
# drain 完後要等一下子讓 pod 在別的 node 產生
$ kubectl drain node-1
node/node-1 already cordoned
evicting pod kube-system/storage-provisioner
evicting pod default/nginx-7c658794b9-zszdd
evicting pod kube-system/coredns-64897985d-dp6lx
pod/storage-provisioner evicted
pod/nginx-7c658794b9-zszdd evicted
pod/coredns-64897985d-dp6lx evicted
node/node-1 evicted
The drain procedure first cordons the Node if you’ve not already placed one manually. It will then evict running Kubernetes workloads by safely rescheduling them to other Nodes in your cluster.
You can shutdown or destroy the Node once the drain’s completed. You’ve freed the Node from its responsibilities to your cluster. The cordon provides an assurance that no new workloads have been scheduled since the drain completed.

檢查

$ kubectl get node

14. Trobleshooting - kubelet 故障

A Kubernetes worker node, named wk8s-node-0 is in state NotReady. Investigate why this is the case, and perform an appropriate steps to bring the node to a Ready stat, ensuring that any changes are made permanent.

You can ssh to the failed node using: ssh wk8s-node-0 You can assume elevated privileges on the node with the following command: sudo -i

解答

#檢查
$ kubectl get nodes
# 連到 wk8s-node-0 這台 node
$ ssh wk8s-node-0
# 提權
$ sudo -i
# 檢查 kubelet 狀態
$ systemctl status kubelet
# 將 kubelet 重啟
$ systemctl start kubelet
# 將 kubelet 設為開機自動啟動
$ systemctl enable kubelet
# 檢查 kubelet 狀態
$ systemctl status kubelet

15. Sidecar

Context Without changing its existing containers, an existing Pod needs to be integrated into Kubernetes’s build-in logging architecture(e.g kubectl logs). Adding a streaming sidecar container is a good and common way to accomplish this requirement.

Task Add a busybox sidecar container to the existing Pod big-corp-app.The new sidecar container has to run the following command:

/bin/sh -c tail -n+1 -f /var/log/big-corp-app.log Use a volume mount named logs to make the file /var/log/big-corp-app.log available to the sidecar container.

Don’t modify the existing container. Don’t modify the path of the log file, both containers must access it at /var/log/bin-corp-app.log

解答

原本的yaml 檔，已經存在

apiVersion: v1
kind: Pod
metadata:
name: big-corp-app
spec:
containers:
- name: big-corp-app
image: busybox
args:
- /bin/sh
- -c
- |
mkdir -p /var/log;
i=0;
while true;
do
echo "$(date) INFO $i" >> /var/log/big-corp-app.log;
i=$((i+1));
sleep 1;
done

測試： 看 log 裡面沒有任何東西 $ k logs big-corp-app

$ tail -n+1 -f aaa
這行命令是 tail 會一直去看 aaa 這個檔案
# -f (file)就是會一直去看檔案
範例
$ tail -n+1 -f aaa
eee
123
haha
yyy
開啟另一個終端機
$ echo eee >> aaa
$ echo 123 >> aaa
$ echo haha >> aaa
$ echo yyy >> aaa

再建一個 Container，名字叫 count-log，要去收集已經存在的 Container log 資訊，答案如下：

答案： K8S DOC 查詢關鍵字 : sidecar yaml 找 Logging Architecture | Kubernetes 這篇 ctrl + f 搜尋 : streaming- 匯出 YAML 檔

$ k get pod big-corp-app -o yaml > sidecar.yaml 編輯 YAML 檔，要刪除原本的 volumes

apiVersion: v1
kind: Pod
metadata:
 annotations:
 kubectl.kubernetes.io/last-applied-configuration: |
 {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"name":"big-corp-app","namespace":"default"},"spec":{"containers":[{"args":["/bin/sh","-c","mkdir -p /var/log;\ni=0;\nwhile true;\ndo\n echo \"$(date) INFO $i\" \u003e\u003e /var/log/big-corp-app.log;\n i=$((i+1));\n sleep 1;\ndone\n"],"image":"quay.io/cloudwalker/busybox","name":"big-corp-app"}]}}
 creationTimestamp: "2022-09-29T14:47:24Z"
 name: big-corp-app
 namespace: default
 resourceVersion: "34289"
 uid: b8598b00-8c2c-4014-80f6-84fa7af04712
spec:
 containers:
 - args:
 - /bin/sh
 - -c
 - |
 mkdir -p /var/log;
 i=0;
 while true;
 do
 echo "$(date) INFO $i" >> /var/log/big-corp-app.log;
 i=$((i+1));
 sleep 1;
 done
 image: quay.io/cloudwalker/busybox 
 imagePullPolicy: Always 
 name: big-corp-app 
 resources: {}
 terminationMessagePath: /dev/termination-log
 terminationMessagePolicy: File 
 volumeMounts:
 - mountPath: /var/run/secrets/kubernetes.io/serviceaccount 
 name: kube-api-access-w2w4r 
 readOnly: true
 volumeMounts: # add
 - name: logs  # add
 mountPath: /var/log  # add
 - name: count-log  # add
 image: busybox  # add
 args: [/bin/sh, -c, 'tail -n+1 -f /var/log/big-corp-app.log']  # add
 volumeMounts: # add
 - name: logs  # add
 mountPath: /var/log  # add
 volumes: # add
 - name: logs  # add
 emptyDir: {} # add
 dnsPolicy: ClusterFirst
 enableServiceLinks: true
 nodeName: w1
 preemptionPolicy: PreemptLowerPriority
 priority: 0
 restartPolicy: Always
 schedulerName: default-scheduler
 securityContext: {}
 serviceAccount: default
 serviceAccountName: default
 terminationGracePeriodSeconds: 30
 tolerations:
 - effect: NoExecute
 key: node.kubernetes.io/not-ready
 operator: Exists
 tolerationSeconds: 300
 - effect: NoExecute
 key: node.kubernetes.io/unreachable
 operator: Exists
 tolerationSeconds: 300
 volumes:
 - name: kube-api-access-w2w4r 
 projected:
 defaultMode: 420
 sources:
 - serviceAccountToken:
 expirationSeconds: 3607
 path: token 
 - configMap:
 items:
 - key: ca.crt 
 path: ca.crt 
 name: kube-root-ca.crt 
 - downwardAPI:
 items:
 - fieldRef:
 apiVersion: v1 
 fieldPath: metadata.namespace 
 path: namespace 
status:
 conditions:
 - lastProbeTime: null
 lastTransitionTime: "2022-09-29T14:47:24Z"
 status: "True"
 type: Initialized
 - lastProbeTime: null
 lastTransitionTime: "2022-09-29T14:47:28Z"
 status: "True"
 type: Ready
 - lastProbeTime: null
 lastTransitionTime: "2022-09-29T14:47:28Z"
 status: "True"
 type: ContainersReady
 - lastProbeTime: null
 lastTransitionTime: "2022-09-29T14:47:24Z"
 status: "True"
 type: PodScheduled
 containerStatuses:
 - containerID: cri-o://a6a13dcf0b11f0e6fc22eb507b23cfde7360995d24af798e7630ba26ddebb6d1
 image: quay.io/cloudwalker/busybox:latest
 imageID: quay.io/cloudwalker/busybox@sha256:f3cfc9d0dbf931d3db4685ec659b7ac68e2a578219da4aae65427886e649b06b
 lastState: {}
 name: big-corp-app
 ready: true
 restartCount: 0
 started: true
 state:
 running:
 startedAt: "2022-09-29T14:47:27Z"
 hostIP: 192.168.61.241
 phase: Running
 podIP: 10.244.1.16
 podIPs:
 - ip: 10.244.1.16
 qosClass: BestEffort
 startTime: "2022-09-29T14:47:24Z"

產生pod $ kubectl replace -f 15.yaml –force

$ kubectl get pod big-corp-app 檢查 $ kubectl logs big-corp-app -c count-log

16.1 Ingress

Create a new nginx Ingress resources as follows:

Name: ping
Namespace: ing-internal
Exposing service hi on path /hi using service port 5678

The avaliability of service hi can be checked using the following command,which should return hi: curl -kL <INTERNAL_IP>/hi

環境準備

安裝 metallb

$ wget -qO - https://raw.githubusercontent.com/metallb/metallb/v0.13.4/config/manifests/metallb-native.yaml | kubectl apply -f -

echo '
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: mlb1
namespace: metallb-system
spec:
addresses:
- 120.96.143.30-120.96.143.50
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: mlb1
namespace: metallb-system' | kubectl apply -f -

$ kubectl get pod -n metallb-system -o wide
$ kubectl get service -n ingress-nginx
$ kubectl get ingressclass
$ k run n1 --image=nginx -n ing-internal -l run=nginx
$ kg all -n ing-internal

安裝 ingress $ wget -qO - https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.3.0/deploy/static/provider/cloud/deploy.yaml | sed 's|name: nginx|name: ig1|g' | kubectl apply -f -

解答

K8S DOC 查詢關鍵字 : ingress
找 Ingress | Kubernetes 這篇
用第一個 YANL 檔編輯 YAML 檔

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ping
annotations:
kubernetes.io/ingress.class: "nginx" # 加這行 ingress 就會有 address
nginx.ingress.kubernetes.io/rewrite-target: /
namespace: ing-internal
spec:
rules:
- http:
paths:
- path: /hi
pathType: Prefix
backend:
service:
name: hi
port:
number: 5678

產生 ingress

$ k apply -f ingress.yaml 測試

$ curl -kL <INTERNAL_IP>/hi

$ curl -w '\n' --resolve k8s.org:80:120.96.143.30 http://k8s.org/hi

16.2 Kubernetes 1.25.0 版的 Ingress 範例操作

$ kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.13.5/config/manifests/metallb-native.yaml

$ echo '
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: mlb1
namespace: metallb-system
spec:
addresses:
- 192.168.61.220-192.168.61.230' | kubectl apply -f -

$ curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash
$ helm repo add nginx-stable https://helm.nginx.com/stable; helm repo update
$ helm search repo nginx-stable
$ helm install ig1 nginx-stable/nginx-ingress --set controller.ingressClass="ig1"
$ wget -qO - http://www.oc99.org/kube/apple-banana.yaml | kubectl apply -f -

$ nano ig1.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ig1
annotations:
kubernetes.io/ingress.class: ig1
spec:
rules:
- host: ig1.k8s.org
http:
paths:
- path: /apple
pathType: Prefix
backend:
service:
name: apple-service
port:
number: 5678
- path: /banana
pathType: Prefix
backend:
service:
name: banana-service
port:
number: 5678

$ kubectl apply -f ig1.yaml
$ kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
ig1 <none> ig1.k8s.org 120.96.143.240 80 25m
$ curl -k -H "Host: ig1.k8s.org" http://120.96.143.240/apple/
apple

17. Etcd （7%）

First, create a snapshot of the existing etcd instance running at https://127.0.0.1:2379, saving the snapshot to /var/lib/backup/etcd-snapshot.db

Creating a snapshot of the given instance is expected to complete in seconds. If the operation seems to hang, something’s likely wrong with your command. Use CTRL+C to cancel the operation and try again.

Next, restore an existing, previous snapshot located at /var/lib/backup/etcd-snapshot-previous.db.

The following TLS certificates/key are supplied for connecting to the server with etcdctl: CA certificate: /opt/KUIN0061/ca.crt Client certificate: /opt/KUIN0061/etcd-client.crt Client key: /opt/KUIN0061/etcd-client.key

環境準備

ubuntu 安裝 etcdctl 命令 $ sudo apt install etcd-client

alpine 安裝 etcdctl 命令

$ sudo apk add etcd-ctl \
--no-cache \
--update-cache \
--allow-untrusted \
--repository http://dl-cdn.alpinelinux.org/alpine/edge/testing

解答

K8S DOC 查詢關鍵字 : etcd backup
點選 Operating etcd clusters for Kubernetes | Kubernetes 這篇文章
進取之後點藍藍的 backup
將 etcd 備份 (考試環境，確認 master node 有無 etcdctl 命令，如沒有則在一開始的終端機做題 )
參考文章 : https://github.com/bmuschko/cka-crash-course/blob/master/exercises/04-etcd-backup-restore/solution/solution.md

# 如果題目沒有給檔案路徑就要到 etcd.yaml 這個檔案去找
$ sudo cat /etc/kubernetes/manifests/etcd.yaml | grep file
- --cert-file=/etc/kubernetes/pki/etcd/server.crt # 這行
- --key-file=/etc/kubernetes/pki/etcd/server.key # 這行
- --peer-cert-file=/etc/kubernetes/pki/etcd/peer.crt
- --peer-key-file=/etc/kubernetes/pki/etcd/peer.key
- --peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt
- --trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt # 這行
seccompProfile:

使用題目給的檔案路徑

$ ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 \
--cacert=/opt/KUIN0061/ca.crt \
--cert=/opt/KUIN0061/etcd-client.crt \
--key=/opt/KUIN0061/etcd-client.key \
snapshot save /var/lib/backup/etcd-snapshot.db

檢查 save 有沒有成功可以下以下這行指令

$ sudo ETCDCTL_API=3 etcdctl --write-out=table snapshot status /var/lib/backup/etcd-snapshot.db

將 etcd 還原 save、restore 在本機操作

$ ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 \
--data-dir /var/lib/etcd-1 \ # 要加 data-dir 這個參數
--cacert=/opt/KUIN0061/ca.crt \
--cert=/opt/KUIN0061/etcd-client.crt \
--key=/opt/KUIN0061/etcd-client.key \
snapshot restore /var/lib/backup/etcd-snapshot-previous.db

修改 yaml 檔要 ssh 進去 master 機

# 修改 etcd.yaml 檔案
$ sudo nano /etc/kubernetes/manifests/etcd.yaml
........
volumeMounts:
- mountPath: /var/lib/etcd
name: etcd-data
- mountPath: /etc/kubernetes/pki/etcd
name: etcd-certs
hostNetwork: true
priorityClassName: system-node-critical
securityContext:
seccompProfile:
type: RuntimeDefault
volumes:
- hostPath:
path: /etc/kubernetes/pki/etcd
type: DirectoryOrCreate
name: etcd-certs
- hostPath:
path: /var/lib/etcd-1 # 修改路徑
type: DirectoryOrCreate
name: etcd-data
status: {}

檢查

$ kubectl get pod

alpine etcd 備份

$ ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 \
--cacert=/opt/KUIN0061/ca.crt \
--cert=/opt/KUIN0061/etcd-client.crt \
--key=/opt/KUIN0061/etcd-client.key \
snapshot save /var/lib/backup/etcd-snapshot.db

將 etcd 還原

$ ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 \
--data-dir /var/lib/etcd-1 \ # 要加 data-dir 這個參數
--cacert=/opt/KUIN0061/ca.crt \
--cert=/opt/KUIN0061/etcd-client.crt \
--key=/opt/KUIN0061/etcd-client.key \
snapshot restore /var/lib/backup/etcd-snapshot.db

# 修改 etcd.yaml 檔案
$ sudo nano /etc/kubernetes/manifests/etcd.yaml
........
volumeMounts:
- mountPath: /var/lib/etcd
name: etcd-data
- mountPath: /etc/kubernetes/pki/etcd
name: etcd-certs
hostNetwork: true
priorityClassName: system-node-critical
securityContext:
seccompProfile:
type: RuntimeDefault
volumes:
- hostPath:
path: /etc/kubernetes/pki/etcd
type: DirectoryOrCreate
name: etcd-certs
- hostPath:
path: /var/lib/etcd-1 # 修改路徑
type: DirectoryOrCreate
name: etcd-data
status: {}

檢查

$ kubectl get pod

tags: `CKA`

J509教室windows電腦設定

Sun, 08 May 2022 03:56:44 +0800

系統環境設定

設定BIOS，實體個人電腦復電自動開機

按下電源鍵，啟動 Windows 主機，於開機畫面出現hp圖示時，按 F10

輸入密碼:

按鍵盤上的向右鍵，選到"Advanced" »> “在Power-On Option"上按Enter鍵

按往下，選到After Power Loss ，利用往左或往右鍵，選為On
- 選好後，按F10儲存
- 再按一次F10

在"Save change and Exit"選項上按Enter

在"Yes"的地方按Enter鍵，儲存設定更改
- 系統會自動重新開機

設定BIOS，啟動CPU的VT-x

選到Security »> System Security »> Virtualization Technology (VTx) » enable »> F10 »> F10 »> Enter鍵

SysReturn Deluxe還原軟體派送

派送的意思，就是直接拷貝發送端的機器裡面所有的資料和環境設定，拷貝過後的機器，基本上裡面的資料和環境設定都會和發送端相同。

==超級注意:派送之前要記好發送端以及接收端的網路IP位址，以及是靜態還是動態，子網路遮罩多少，最後還有gateway也要記起來==

拔掉教室右前方導師機左邊型號為:Zyxel GS1900-48的Switch上面編號1的孔的網路線
確認教室的windows電腦不能上網
將發送端的windows電腦重新開機
- 打開命令提示字元(CMD)
- 執行shutdown -r -t 1 #電腦將在1秒後重新開機
看完內容再動作
- 在電腦開機畫面跑到DHCP時，按ESC鍵
- 看到螢幕顯示變成藍畫面時，立刻按下HOME鍵
  
  HOME鍵在Delete鍵的右上方
輸入還原密碼: cc8050
利用鍵盤上的向右鍵，移動到”拷貝工具"，然後按下Enter鍵
按下Enter鍵後，畫面會變黑色的，左上角會有數字在倒數，是正常的
用滑鼠選擇發送端後，不要按確定，直接點選上方的PXE伺服器
輸入本機IP位址: 192.168.1.?(依當時發送端決定)
```
IP位址，192.168.1.?~192.168.1.?(依當時要還原的機器IP範圍是多少，
假設要還原192.168.1.1~192.168.1.3，這3台電腦，
那IP位址就是填192.168.1.1~192.168.1.3
```
子網路遮罩: 255.255.255.0 以上都填好後。移動滑鼠按下"啟動"
- 再次確認
  - 是否有拔掉把教室右前方導師機左邊型號為:Zyxel GS1900-48的Switch上面編號1的埠的網路線
  - 教室的電腦都不能上網
按確定
此時會進到對拷的設定，可以看到教室裡的所有電腦，哪些是離線的，哪些有上線，可以透過滑鼠在標題中不同的欄位上按一下調整順序
將接收端的windows電腦，進行重新開機
- 注意開機畫面要有以下畫面，才會進到對拷的設定
```
 Loading NetCopy from PXE...
Loading bzImage......
Loading linux.gz...................Ready
```
如果成功進到對拷畫面可跳過這個步驟，如果沒有以上畫面，直接進到windows的桌面，此時要再進行一次重新開機，這次要進到BIOS做設定
- 一樣在開機畫面出現hp時，就可以多按幾次F10
- 進去以後按向右鍵，選到"advanced"後，按向下鍵，選到"Device Option"，按Enter鍵
- 按向下鍵，選到"NIC PXE Option ROM Download"，按向右鍵將Disabled，改成Enabled狀態
- 選好後，按F10儲存，再按一次F10 在"Yes"的地方按Enter鍵，儲存設定更改
- 系統會自動重新開機
回到發送端電腦，確定所有接收端都有進到對拷設定，在狀態列顯示線上，代表等等可以順利進行對拷設定
可以用滑鼠在要設定的電腦連點兩下，設定IP位址為"自動取得IP位址"，DNS設定資訊，或其他進階設定，如果陳老師沒有特別指示，就不作任何更動，以上設定好後，移動滑鼠到右下角按確定
將要對拷的主機都選起來後，按下一步，確定接收端的主機都有選到後，按是
選擇要對拷的資料，這邊就把上面3個受保護的磁碟都選起來，可視情況選擇要對拷的磁碟，選好後就按下一步
這時候對拷就會開始，如果發現有跳紅紅的，可以不用理他，等下對考完後，去接收端檢查設定有沒有對拷過去即可
對拷完後，發送端的主機和所有接收端的主機都會自動重新開機，開機完後，接收端的螢幕右上角會出現以下畫面
- 這時候，按Ctrl+Alt+Delete解鎖螢幕畫面，用student帳號登入，密碼為:student
- 登入後，螢幕會出現一個進度條，等他跑完後，系統會自動重新開機
接收端的電腦都重新開機兩次後，就可以開始檢查，我們做過的環境設定和資料是否齊全。

檢查發送端和接收端的所有電腦

移除SysReturn Deluxe還原軟體

點左下角收尋，輸入"sys"，找到SysReturn Deluxe還原軟體，按開啟
輸入密碼: cc8050，按登入
點右下角的3個點，選移除
選"保留目前狀態後刪除"，按套用
是否立即啟動，選"是"

電腦重新開機 »> 等待系統更新起始點 »> 登入student帳號後 »> 系統會刪除還原軟體 »> 再次重新開機 »> 再次登入student ，還原軟體刪除成功

檢查還原軟體是否被移除，在左下角收尋打"sys"，沒有顯示出來，代表刪除成功

設定IPV4為靜態IP位址

打開"檔案總管"，在左側選項的最下面，找到"網路"，滑鼠右鍵點它，選"內容"
選擇左上方的"變更介面卡設定"
在"Ethernet"那張網卡上，滑鼠右鍵，點"內容"，
往下滑，找到"Internet Protocol Version 4(TCP/IPV4)"，移動滑鼠選它，點"內容"
點選"使用下列的IP位址(s)": IP位址(I) : 192.168.1.25 (本次範例為192.168.1.25，要自己看當時接收端的IP為只要設多少) 子網路遮罩(U) : 255.255.255.0 (J509教室每台電腦都一樣) 預設閘道(D) : 192.168.1.1 (J509教室每台電腦都一樣) (可以在J509教室正常運作的其他台電腦，打開CMD，打ipconfig，就可以看到子網路遮罩和預設閘道是設多少) 以上都設好後，按"確定"
再按"關閉"
檢查有沒有成功把IPV4設為靜態IP位址打開CMD，輸入ipconfig 可以看到，乙太網路卡 Ethernet那張網卡的IPV4位址，已成功設定為固定的IP(192.168.1.25)，子網路遮罩(255.255.255.0)和預設閘道(192.168.1.1)
檢查網路內網跟外網有沒有都通
1. 先ping本機看有沒有通 ping 127.0.0.1 看到封包 : 已傳送 = 4 ，已收到 = 4, 已遺失 = 0 ，代表有通
2. 再ping教室其他台電腦的IP看有沒有通 ping 192.168.1.5 (不一定要跟我一樣ping 192.168.1.5，只要是在J509教室的電腦都可以) 看到封包 : 已傳送 = 4 ，已收到 = 4, 已遺失 = 0 ，代表有通
3. 再ping預設閘道看有沒有通 ping 192.168.1.1 看到封包 : 已傳送 = 4 ，已收到 = 4, 已遺失 = 0 ，代表有通
4. 再ping Google在美國的一台電腦有沒有通，有通代表可以上網 ping 8.8.8.8 看到封包 : 已傳送 = 4 ，已收到 = 4, 已遺失 = 0 ，代表有通
5. 測一下DNS有沒有正常 ping www.google.com 看到封包 : 已傳送 = 4 ，已收到 = 4, 已遺失 = 0 ，代表有通

檢查使用者帳號和群組

先檢查接收端的主機，一台一台檢查
使用student帳號登入接收端的Windows帳號
注意，對拷後的接收端，會複製發送端的主控台，所以不能用桌面上的主控台1.msc
- 打開桌面上的主控台1.msc
- 可以發現在本機與使用者和群組(NC1848394C)，這個選項後的括號，是發送端的電腦名稱，以本次範例就是NC1848394C，所以再次證明，這個主控台不是本機
- 此時先按右上角的X，關閉視窗，在桌面的主控台1.msc圖示上，按滑鼠右鍵，選擇"刪除"，(如果跳出是否要儲存就按"否")。
- 在資源回收桶上按右鍵，選擇"清理資源回收桶"
- 系統會再跳出一次訊息，問你是不是真的要刪除，選"是"，我們要刪除它。
- 在左下角的收尋，收尋"mmc"，在右邊的選項中，選"以系統管理員身分執行(A)"
- 在點左上角檔案，選擇"新增/移除嵌入式管理單元"
- 點選左邊的"本機使用者和群組"後，點中間的"新增(A)>"
- 選"本機電腦"後，按完成
- 點左邊選項中的"在本機電腦上具有進階安全性的Windows Defender防火牆"後，再按"新增(A)>"
- 選"本機電腦"後，按完成
- 點左邊選項中的"服務"後，按"新增(A)>"
- 選"本機電腦"後，按完成
- 點左邊"本機使用者和群組(本機) »> 使用者"在bigred那欄，按滑鼠右鍵，點內容
- 檢查"使用者不能變更密碼"和"密碼永久有效"兩個有沒有都有打勾
- 點"成員隸屬"，檢查有沒有"Administrator"和"Remote Desktop Users"
- 在使用者，gbean那欄，按滑鼠右鍵，點內容
- 檢查"使用者不能變更密碼"和"密碼永久有效"兩個有沒有都有打勾
- 點"成員隸屬"，檢查有沒有"Remote Desktop Users"
- 在使用者，rbean那欄，按滑鼠右鍵，點內容
- 檢查"使用者不能變更密碼"和"密碼永久有效"兩個有沒有都有打勾
- 點"成員隸屬"，檢查有沒有"Remote Desktop Users"
- 在使用者，ybean那欄，按滑鼠右鍵，點內容
- 檢查"密碼永久有效"有沒有都有打勾
- 點"成員隸屬"，檢查有沒有"Remote Desktop Users"
檢查群組，用滑鼠點一下，主控台根目錄 »> 本機使用者和群組 »> 群組
- 在"Administrators"那欄，滑鼠右鍵點內容
- 再確認一次有沒有bigred
- 在"Remote Desktop Users"那欄，滑鼠右鍵點內容
- 再次確認"Remote Desktop Users"群組裡有沒有bigred、gbean和rbean 3位使用者
以上都確認好後，
- 按左上角"檔案"，再按"另存新檔" 選桌面
- 點"桌面"，檔名設為:主控台.msc，再按"存檔"
- 回到windows桌面，看看有沒有存檔成功

檢查OpenSSH Server

檢查OpenSSH Server是否為Windows內鍵安裝的
- 左下角開始，點設定
- 選"應用程式"
- 點"選用功能"
- 確認安裝OpenSSH Server
- 檢查OpenSSH Server功能有沒有正常
  - 打開CMD
  - 打ipconfig,確認IP位址
  - 到其他台電腦開一個CMD，執行ssh bigred@192.168.1.25 打yes，再打密碼:1大兩小3大兩驚嘆
    
    進去後打個dir，確認一下可以動作
  - exit離開
  - 再執行ssh rbean@192.168.1.25，密碼:srerbean
  - 進去後打個dir，確認一下可以動作
  - exit離開
  - 再執行ssh gbean@192.168.1.25，密碼:sregbean
  - 進去後打個dir，確認一下可以動作
  - exit離開
  - 再執行ssh ybean@192.168.1.25，密碼:sreybean
  - 進去後打個dir，確認一下可以動作
  - exit離開

以上都可以正常連線，代表OpenSSH Server OK

檢查RDP(Remote Desktop Protocol)通訊協定

打開檔案總管，到以下目錄，C:\jetpack\RDPWrap，找到install.bat這個批次檔，按滑鼠右鍵，選"以系統管理員身分執行"

等到CMD裡面跑完後，跳出以下視窗，注意內容要看到:

Diagnostics
Wrapper state: Installed ver 1.5.0.0
Service state: Running ver 10.0.19041.1620
Listener state: Listening [fully supported]

一定要有4個綠色的英文，還有注意port號:2599

檢查電源選項(bigred、rbean、gbean和ybean都要做)

桌面滑鼠右鍵 -> 顯示設定 -> 電源與睡眠螢幕調15分鐘後關閉，睡眠調成永不

檢查關閉關機選項(bigred、rbean、gbean和ybean都要做)

按左下角開始，點student，登出student帳號
- 用bigred帳號登入，按左下角開始，再按電源，查看關機鍵和睡眠鍵有沒有拔掉，有拔掉的話，會顯示"目前沒有可用的電源選項"
- 用滑鼠左鍵點兩下桌面上的"VMware Workstation 16 player"
- 查看是否只有這個視窗
- 若出現以下畫面，代表失敗。
用gbean、rbean和ybean帳號，重複做以上動作:登出帳號 »> 看關機鍵和睡眠鍵有沒有拔掉 »> 用滑鼠左鍵點兩下桌面上的"VMware Workstation 16 player" ，看是否有跳出其他視窗

檢查Windows關閉更新

選"更新與安全性"
確認無法更新

檢查CMD Autorun(bigred、rbean、gbean和ybean都要做)

打開CMD 輸入: echo %boss%

檢察關閉IPV6

打開"檔案總管"，在左側選項的最下面，找到"網路"，滑鼠右鍵點它，選"內容"
選擇左上方的"變更介面卡設定"
在"Ethernet"那張網卡上，滑鼠右鍵，點"內容"，
往下滑，確認"Internet Protocol Version 6 (TCP/IP)“沒有打勾

==檢查防火牆關閉(很重要!!!沒關的話，落地雲會無法連線!!!)==

打開"檔案總管”，在左側選項的最下面，找到"網路"，滑鼠右鍵點它，選"內容"
點左下角的"Windows Defender防火牆"，確認都是紅色的XX

使用者設定(bigred、rbean、gbean和ybean都要做)

打開"檔案總管"，點上面的"檢視"，將附檔名打勾，再點旁邊的選項

2. 點選"檢視"，往下滑，將"顯示所有資料夾"打勾

將工作列沒有要用到的應用程式取消訂選
最後只保留以下這3個，檔案總管、CMD和Google Chrome
打開CMD，點左上角
- 點預設值
- 將字型調成20
- 選"色彩"，將螢幕背景選成白色
- 螢幕文字選成黑色，好了以後按確定
- 關掉CMD，再打開一次，檢查有沒有更改成功

所有系統設定和環境都確定好後，再把SysReturn Deluxe還原軟體安裝回來

打開"檔案總管"，到以下目錄，C:\Tools\SysReturn Deluxe V10.0.011-20201130\Client Setup(W525) 找到Setup.bat，滑鼠右鍵點它，選"以系統管理員身分執行"
使用預設"中文繁體"，按確定
授權合約，按下一步
使用預設安裝路徑，按下一步
都跟預設一樣，不要動，直接按下一步
將Windows 更新安裝策略選為"保持原來設定"，按下一步
選擇要保護的磁碟，把D槽以外的都選起來，好了後按下一步
選擇還原模式，選"暫不還原"，按下一步
參數設定，選使用指定密碼，密碼: cc8050，按下一步
連接設定，使用預設port:7910，其他不要動，按下一步
按否
參數確認，注意還原模式要:妹次啟動都保留，密碼:cc8050，好了以後按下一步就會開始安裝
正在安裝中…
按是，重新開機
重新開機後，用student帳號登入windows電腦
在左下角收尋打sys，打開SysReturn Deluxe還原軟體
填註冊名稱: mcu.edu.tw 產品序號: B35AA14BA346CFFDCF401DBD1 (在C:\Tools\SysReturn Deluxe V10.0.011-20201130\Client Setup(W525)\10.bat.txt 裡面有)
好了以後，打開SysReturn Deluxe還原軟體，輸入密碼，就會看到Root，系統自動幫我們現在這個環境建好的還原點
點上方選項的還原點管理
按新增還原點
在名稱打: SRE第四期，按確定，系統就會開始把現在的環境和設定，建立一個新的還原點
儲存成功，按確定

Antony 筆記：技術原理與實務經驗 – 學習筆記

生成式 AI 課程

環境準備

Hardware

安裝軟體

實作 : 利用 PyTorch 建立一個 Word2Vec 詞嵌入模型

範例 2

範例 3

範例 4

範例 5

範例 6

範例 7 : 實作 LSTM 模型來處理給定的文本

範例 7

範例 8

範例 9

範例 10

Build HA K8S - 3M

1. 安裝 kubectl , kubeadm , kubelet

2. kube-vip (set up m1 node)

3. 建立 k8s 系統

命令式

設定檔方式

4. 將 bigred 設成 K8S 管理者

5. 設定 K8S Master 可以執行 Pod

6. 安裝 Flannel 網路套件

7. join m2 and m3 control-plane node

create control-plane install command

8. set up m2 node

9. set up m3 node

10. 檢視 Kubernetes 系統狀態

開始建置 Kubernetes Worker

w1, w2 安裝 kubelet 及 kubeadm 命令

w1, w2 加入 K8S Cluster

設定 K8S Worker 標籤

測試

重新建立損壞 master

在 Alpine Linux 安裝 etcd-ctl 命令套件檔

清除損壞的 master node

找到那一台 壞掉 master node 的 ID

Join a Node to a Kubernetes Cluster

Get Kubeadm Join Command for Master Node

Joining Master Node to Kubernetes CLuster

如果不是全新的機器要把舊的檔案，目錄刪除

加到 kube-vip

tags: K8S-陳老師-第6期

CKA

考試環境圖

1. Scale

環境準備

解答

2. NetworkPolicy （7%）

解答

驗證

2.2 NetworkPolicy

解答

3. Multi-container

解答

4. CPU

環境準備

解答

5. NodeSelector

解答

6. Kubernetes 升級

7. Service

解答

8. RBAC

解答

9. Check Ready Node

解答

10. PVC

解答

11. Pod Logs

解答

12. PV

題型 1: 解答

題型 2: 解答

13. cordon & drain

解答

檢查

14. Trobleshooting - kubelet 故障

找到那一台壞掉 master node 的 ID

tags: `K8S-陳老師-第6期`

tags: `CKA`

tags: `J509教室`