前言
先前有研究 Windows Desktop 上的 Kubernetes 整合 jenkins，這次來試著用Jenkins 整合 Linux 的 Kubernetes。
預先準備
- Ubuntu 上己安裝 Docker、 Kubernetes、 Registy
- Jenkins
- Next JS 專案，裡面並寫好 yaml 檔
首先，要先安裝 Kubernetes Dashboard及產生 ServiceAccount 及 取得Token (這個token 是要拿去給jenkins 用的)
安裝 Kubernetes Dashboard
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml
建立 admin user
sudo vim dashboard-adminuser.yaml
建立 admin user
apiVersion: v1
kind: ServiceAccount
metadata:
name: admin-user
namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: admin-user
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: admin-user
namespace: kubernetes-dashboard
create service account
kubectl apply -f dashboard-adminuser.yaml
get admin token
kubectl -n kubernetes-dashboard create token admin-user --duration=876000h
啟動Proxy
kubectl proxy
此時訪問以下連結即可登入Dashboard
http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/login
安裝 Jenkins
參考先前撰寫的文章
在Jenkins 容器中中安裝 kubectl
docker exec -it -uroot jenkins bash // 進入jenkins 容器中
curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.29.1/bin/linux/amd64/kubectl // 下載 kubectl
chmod +x ./kubectl // 給予權限
mv ./kubectl /usr/local/bin/kubectl // 複製到系統環境資料夾
kubectl version --client // 查詢版本
P.S. Unbuntu 和 Jenkins 容器的 kubectl 版本最好一致
複製 kubectl 設定至 Jenkins 容器中 (需要逐行執行)
docker exec -it -uroot jenkins /bin/bash
mkdir -p /.kube
exit
docker cp ~/.kube/config jenkins:/root/.kube
Jenkins 透過 Kubernetes CLI 連線 kubernetes
- 要想把jenkins關聯到Kubernetes中需要安裝幾個最件，打開 系統管理 > 套件管理
- Kubernetes plugin
- Kubernetes CLI Plugin
- set up cloud
- set Disable https certificate check => true
4. Kubernetes URL =>
Ubuntu 透過可以 kubectl cluster-info 查出 Url
https://192.168.50.50:6443 // 192.168.50.50是我家的Ubuntu 內網的主機IP
- Jenkins URL Jenkins URL:http://host.docker.internal:8080/ Jenkins tunnel:https://192.168.50.50:50000
- Credentials =>
- Copy dashboard token to here
撰寫 Jenkins pipeline
properties([pipelineTriggers([githubPush()])])
pipeline {
agent any
environment {
tag = ':latest'
imageShortName = 'k8s-next-ec'
imageName = "${imageShortName}${tag}"
containerName = "${imageShortName}-1"
dockerfile = "./Dockerfile"
registryUrl = "192.168.50.50:5000"
registry = "${registryUrl}/${imageShortName}"
}
stages {
stage("GitHub Pull") {
steps {
git branch: 'main',
credentialsId: 'e85233ad-a3c5-448b-a6ea-9f53e4f9b3f1',
url: '[email protected]:markku636/ec.git/'
}
}
stage("Building Docker Image") {
steps {
script {
dockerImage = docker.build "$registry${tag}"
}
}
}
stage("Deploying to Registry Server") {
steps {
script {
docker.withRegistry("","") {
dockerImage.push("latest")
}
}
}
}
stage("Cleaning Up") {
steps {
sleep(time: 3, unit: "SECONDS")
sh "docker rmi --force $registry:latest"
}
}
stage("Deply") {
steps {
withKubeConfig([credentialsId: 'k8s-secret', serverUrl: 'https://192.168.50.50:6443']) {
sh 'kubectl apply -f ./next-js-deployment.yaml'
sh 'kubectl rollout restart deployment/k8s-next-ec'
}
}
}
}
}
在 Github next-ec 專案裡建立 Kubernetes 的 YAML 佈署設定檔(next-js-deployment.yaml)
apiVersion: apps/v1
kind: Deployment
metadata:
name: k8s-next-ec
labels:
app: k8s-next-ec
spec:
selector:
matchLabels:
app: k8s-next-ec
tier: web
template:
metadata:
labels:
app: k8s-next-ec
tier: web
spec:
containers:
- name: k8s-next-ec-app
image: 192.168.50.50:5000/k8s-next-ec:latest
ports:
- containerPort: 3000
---
apiVersion: v1
kind: Service
metadata:
name: k8s-next-ec
labels:
app: k8s-next-ec
spec:
selector:
app: k8s-next-ec
type: NodePort
ports:
- name: http
protocol: TCP
port: 3000
targetPort: 3000
nodePort: 30066
此時執行 Jenkins 執行建置，應該己經可以看到成功。
補充 - 如果遇到權限不足，出現以下錯誤
ERROR: permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Head "http://%2Fvar%2Frun%2Fdocker.sock/_ping": dial unix /var/run/docker.sock: connect: permission denied 短解
sudo chmod 777 /var/run/docker.sock
長解
sudo nano /etc/systemd/system/docker-sock-permission.service
[Unit]
Description=Set permission on /var/run/docker.sock
After=docker.service
Requires=docker.service
[Service]
Type=oneshot
ExecStart=/bin/chmod 777 /var/run/docker.sock
RemainAfterExit=true
[Install]
WantedBy=multi-user.target
重新載入 systemd 並啟用服務
sudo systemctl daemon-reexec
sudo systemctl daemon-reload
sudo systemctl enable docker-sock-permission.service
sudo systemctl start docker-sock-permission.service
sudo systemctl status docker-sock-permission.service
留言