使用Jenkins 部署 Next.js 至 Ubuntu 的 Kubernetes

在 ChatGPT 開啟在 Claude 開啟

Jenkins 整合 Ubuntu Kubernetes，透過 Kubernetes CLI Plugin 連接，部署 GitHub 上的 Next.js 專案。需安裝 Kubernetes Dashboard、ServiceAccount、Admin Token，並將 kubectl 設定複製到 Jenkins 容器。

前言

先前有研究 Windows Desktop 上的 Kubernetes 整合 jenkins，這次來試著用Jenkins 整合 Linux 的 Kubernetes。

預先準備

Ubuntu 上己安裝 Docker、 Kubernetes、 Registy
Jenkins
Next JS 專案，裡面並寫好 yaml 檔

首先，要先安裝 Kubernetes Dashboard及產生 ServiceAccount 及取得Token (這個token 是要拿去給jenkins 用的)

安裝 Kubernetes Dashboard

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml

建立 admin user

sudo vim dashboard-adminuser.yaml

建立 admin user

apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: admin-user
    namespace: kubernetes-dashboard

create service account

kubectl apply -f dashboard-adminuser.yaml

get admin token

kubectl -n kubernetes-dashboard create token admin-user  --duration=876000h

啟動Proxy

kubectl proxy

此時訪問以下連結即可登入Dashboard

http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/login

安裝 Jenkins

參考先前撰寫的文章

在Jenkins 容器中中安裝 kubectl

docker exec -it -uroot jenkins bash // 進入jenkins 容器中
curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.29.1/bin/linux/amd64/kubectl // 下載 kubectl
chmod +x ./kubectl // 給予權限
mv ./kubectl /usr/local/bin/kubectl // 複製到系統環境資料夾
kubectl version --client // 查詢版本

P.S. Unbuntu 和 Jenkins 容器的 kubectl 版本最好一致

複製 kubectl 設定至 Jenkins 容器中 (需要逐行執行)

docker exec -it -uroot jenkins /bin/bash 
mkdir -p /.kube
exit 
docker cp ~/.kube/config jenkins:/root/.kube

Jenkins 透過 Kubernetes CLI 連線 kubernetes

要想把jenkins關聯到Kubernetes中需要安裝幾個最件，打開系統管理 > 套件管理

Kubernetes plugin
Kubernetes CLI Plugin

set up cloud
set Disable https certificate check => true

4. Kubernetes URL => Ubuntu 透過可以 kubectl cluster-info 查出 Url ![disable http certificate check](/content/markku/posts/jenkins-deploy-nextjs-to-ubuntu-kubernetes/images/kubectl cluster-info.png.png)

https://192.168.50.50:6443  // 192.168.50.50是我家的Ubuntu 內網的主機IP

Jenkins URL Jenkins URL:http://host.docker.internal:8080/ Jenkins tunnel:https://192.168.50.50:50000
Credentials =>
Copy dashboard token to here

撰寫 Jenkins pipeline

properties([pipelineTriggers([githubPush()])])
pipeline {
    agent any 

    environment {
        tag = ':latest'
        imageShortName = 'k8s-next-ec'
        imageName = "${imageShortName}${tag}"
        containerName = "${imageShortName}-1"        
        dockerfile = "./Dockerfile"        
        registryUrl = "192.168.50.50:5000"
        registry = "${registryUrl}/${imageShortName}"
        
    }   
   
    stages {
        stage("GitHub Pull") {
             steps {
                git branch: 'main', 
                credentialsId: 'e85233ad-a3c5-448b-a6ea-9f53e4f9b3f1', 
                url:  '[email protected]:markku636/ec.git/'
            }
            
        }
        
          stage("Building Docker Image") {
			steps {
				script {
					dockerImage = docker.build "$registry${tag}"
				}
			}
		}
		
		stage("Deploying to Registry Server") {
			steps {
			    script {
				    docker.withRegistry("","") {
					  dockerImage.push("latest")
					}
			    }
			}
		}
        
        stage("Cleaning Up") {
            steps {
                sleep(time: 3, unit: "SECONDS")

                sh "docker rmi --force $registry:latest"
            }
        }                   
        
        stage("Deply") {
             steps {
                    withKubeConfig([credentialsId: 'k8s-secret', serverUrl: 'https://192.168.50.50:6443']) {                     
                     sh 'kubectl apply -f ./next-js-deployment.yaml'
					 sh 'kubectl rollout restart deployment/k8s-next-ec'
                    }
                
             }
        }                         
    }
}

在 Github next-ec 專案裡建立 Kubernetes 的 YAML 佈署設定檔(next-js-deployment.yaml)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: k8s-next-ec
  labels:
    app: k8s-next-ec
spec:
  selector:
    matchLabels:
      app: k8s-next-ec
      tier: web
  template:
    metadata:
      labels:
        app: k8s-next-ec
        tier: web
    spec:
      containers:
      - name: k8s-next-ec-app
        image: 192.168.50.50:5000/k8s-next-ec:latest
        ports:
        - containerPort: 3000            
---

apiVersion: v1
kind: Service
metadata:
  name: k8s-next-ec
  labels:
    app: k8s-next-ec
spec:
  selector:
    app: k8s-next-ec
  type: NodePort
  ports:
    - name: http
      protocol: TCP
      port: 3000
      targetPort: 3000
      nodePort: 30066

此時執行 Jenkins 執行建置，應該己經可以看到成功。 Jenkins 執行結果

補充 - 如果遇到權限不足，出現以下錯誤

ERROR: permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Head "http://%2Fvar%2Frun%2Fdocker.sock/_ping": dial unix /var/run/docker.sock: connect: permission denied 短解

sudo chmod 777 /var/run/docker.sock

長解

sudo nano /etc/systemd/system/docker-sock-permission.service

[Unit]
Description=Set permission on /var/run/docker.sock
After=docker.service
Requires=docker.service

[Service]
Type=oneshot
ExecStart=/bin/chmod 777 /var/run/docker.sock
RemainAfterExit=true

[Install]
WantedBy=multi-user.target

重新載入 systemd 並啟用服務

sudo systemctl daemon-reexec
sudo systemctl daemon-reload
sudo systemctl enable docker-sock-permission.service
sudo systemctl start docker-sock-permission.service
sudo systemctl status docker-sock-permission.service