前言

先前有研究 Windows Desktop 上的 Kubernetes 整合 jenkins，這次來試著用Jenkins 整合 Linux 的 Kubernetes。

預先準備

Ubuntu 上己安裝 Docker、 Kubernetes、 Registy
Jenkins
Next JS 專案，裡面並寫好 yaml 檔

首先，要先安裝 Kubernetes Dashboard及產生 ServiceAccount 及取得Token (這個token 是要拿去給jenkins 用的)

安裝 Kubernetes Dashboard

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml

建立 admin user

sudo vim dashboard-adminuser.yaml

建立 admin user

apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: admin-user
    namespace: kubernetes-dashboard

create service account

kubectl apply -f dashboard-adminuser.yaml

get admin token

kubectl -n kubernetes-dashboard create token admin-user  --duration=876000h

啟動Proxy

kubectl proxy

此時訪問以下連結即可登入Dashboard

http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/login

在Jenkins 容器中中安裝 kubectl

docker exec -it -uroot jenkins bash // 進入jenkins 容器中
curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.29.1/bin/linux/amd64/kubectl // 下載 kubectl
chmod +x ./kubectl // 給予權限
mv ./kubectl /usr/local/bin/kubectl // 複製到系統環境資料夾
kubectl version --client // 查詢版本

P.S. Unbuntu 和 Jenkins 容器的 kubectl 版本最好一致

複製 kubectl 設定至 Jenkins 容器中 (需要逐行執行)

docker exec -it -uroot jenkins /bin/bash 
mkdir -p /.kube
exit 
docker cp ~/.kube/config jenkins:/root/.kube

Jenkins 透過 Kubernetes CLI 連線 kubernetes

要想把jenkins關聯到Kubernetes中需要安裝幾個最件，打開系統管理 > 套件管理

Kubernetes plugin
Kubernetes CLI Plugin

install kubernetes plugin

set up cloud

set up cloud
set Disable https certificate check => true

4. Kubernetes URL => Ubuntu 透過可以 kubectl cluster-info 查出 Url ![disable http certificate check](kubectl cluster-info.png.png)

https://192.168.50.50:6443  // 192.168.50.50是我家的Ubuntu 內網的主機IP

Jenkins URL Jenkins URL:http://host.docker.internal:8080/ Jenkins tunnel:https://192.168.50.50:50000
Credentials =>

add public key to jenkins
Copy dashboard token to here

Copy dashboard token to here

撰寫 Jenkins pipeline

properties([pipelineTriggers([githubPush()])])
pipeline {
    agent any 

    environment {
        tag = ':latest'
        imageShortName = 'k8s-next-ec'
        imageName = "${imageShortName}${tag}"
        containerName = "${imageShortName}-1"        
        dockerfile = "./Dockerfile"        
        registryUrl = "192.168.50.50:5000"
        registry = "${registryUrl}/${imageShortName}"
        
    }   
   
    stages {
        stage("GitHub Pull") {
             steps {
                git branch: 'main', 
                credentialsId: 'e85233ad-a3c5-448b-a6ea-9f53e4f9b3f1', 
                url:  '[email protected]:markku636/ec.git/'
            }
            
        }
        
          stage("Building Docker Image") {
            steps {
                script {
                    dockerImage = docker.build "$registry${tag}"
                }
            }
        }
        
        stage("Deploying to Registry Server") {
            steps {
                script {
                    docker.withRegistry("","") {
                      dockerImage.push("latest")
                    }
                }
            }
        }
        
        stage("Cleaning Up") {
            steps {
                sleep(time: 3, unit: "SECONDS")

                sh "docker rmi --force $registry:latest"
            }
        }                   
        
        stage("Deply") {
             steps {
                    withKubeConfig([credentialsId: 'k8s-secret', serverUrl: 'https://192.168.50.50:6443']) {                     
                     sh 'kubectl apply -f ./next-js-deployment.yaml'
                     sh 'kubectl rollout restart deployment/k8s-next-ec'
                    }
                
             }
        }                         
    }
}

在 Github next-ec 專案裡建立 Kubernetes 的 YAML 佈署設定檔(next-js-deployment.yaml)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: k8s-next-ec
  labels:
    app: k8s-next-ec
spec:
  selector:
    matchLabels:
      app: k8s-next-ec
      tier: web
  template:
    metadata:
      labels:
        app: k8s-next-ec
        tier: web
    spec:
      containers:
      - name: k8s-next-ec-app
        image: 192.168.50.50:5000/k8s-next-ec:latest
        ports:
        - containerPort: 3000            
---

apiVersion: v1
kind: Service
metadata:
  name: k8s-next-ec
  labels:
    app: k8s-next-ec
spec:
  selector:
    app: k8s-next-ec
  type: NodePort
  ports:
    - name: http
      protocol: TCP
      port: 3000
      targetPort: 3000
      nodePort: 30066

此時執行 Jenkins 執行建置，應該己經可以看到成功。

ERROR: permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Head “http://%2Fvar%2Frun%2Fdocker.sock/_ping”: dial unix /var/run/docker.sock: connect: permission denied 短解

sudo chmod 777 /var/run/docker.sock

長解

sudo nano /etc/systemd/system/docker-sock-permission.service

[Unit]
Description=Set permission on /var/run/docker.sock
After=docker.service
Requires=docker.service

[Service]
Type=oneshot
ExecStart=/bin/chmod 777 /var/run/docker.sock
RemainAfterExit=true

[Install]
WantedBy=multi-user.target

重新載入 systemd 並啟用服務

sudo systemctl daemon-reexec
sudo systemctl daemon-reload
sudo systemctl enable docker-sock-permission.service
sudo systemctl start docker-sock-permission.service
sudo systemctl status docker-sock-permission.service