您现在的位置是：首页 >其他 >Kubernetes配置Jenkins Slave网站首页其他

Kubernetes配置Jenkins Slave

眼眸流转 2024-10-27 00:01:04
简介Kubernetes配置Jenkins Slave

Kubernetes配置Jenkins Slave

部署在kubernetes集群内

1、部署jenkins

1.1、命名空间

apiVersion: v1
kind: Namespace
metadata:
  name: jenkinsci
  labels:
    app: jenkinsci

1.2、Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: jenkinsci
  namespace: jenkinsci
  labels:
    app: jenkinsci
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jenkinsci
  template:
    metadata:
      labels:
        app: jenkinsci
    spec:
      # 增加污点容忍，可以在具有NoSchedule污点节点上部署
      tolerations:
        - key: node-role.kubernetes.io/master
          operator: Exists
          effect: NoSchedule
        - key: node-role.kubernetes.io/control-plane
          operator: Exists
          effect: NoSchedule
      # 选择具有jenkinsci标签的节点部署
      nodeSelector:
        node-role: jenkinsci
      containers:
        - name: jenkinsci
          # 镜像制作详情https://blog.csdn.net/gyfghh/article/details/130377218
          image: registry.cn-hangzhou.aliyuncs.com/ialso/jenkinsci:0.1
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8080
              name: web
            - containerPort: 50000
              name: jnlp
          volumeMounts:
            - name: timezone
              mountPath: /etc/localtime
            - name: docker
              mountPath: /var/run/docker.sock
            - name: jenkinsci-data
              mountPath: /root/.jenkins
      volumes:
        # 时间处理
        - name: timezone
          hostPath:
            path: /usr/share/zoneinfo/Asia/Shanghai
        # 使用宿主机docker
        - name: docker
          hostPath:
            path: /var/run/docker.sock
        # jenkins数据存放位置
        - name: jenkinsci-data
          hostPath:
            path: /root/.jenkins

1.3、service

apiVersion: v1
kind: Service
metadata:
  name: jenkinsci
  namespace: jenkinsci
spec:
  selector:
    app: jenkinsci
  ports:
    - name: web
      port: 80
      targetPort: 8080
    - name: jnlp
      port: 50000
      targetPort: 50000
  type: ClusterIP

1.4、配置外部访问

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-jenkinsci
  namespace: jenkinsci
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  # 转发规则
  rules:
    - host: jenkins.ialso.cn
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: jenkinsci
                port:
                  number: 80

1.5、数据文件

文件中已包含所有所需插件

链接：https://pan.baidu.com/s/1feLvKsWGK-wMu7Bi6rl_0w?pwd=atqe, 解压缩放入/root/.jenkins

账户：Jenkinsci 密码：xumeng2233.

1.6、配置jenkins localtion

Dashboard->Manage Jenkins->Configure System

需根据jenkinsci在kubernetes中访问url对jenkins进行配置

# http://{service-name}.{name-space}.svc.cluster.local
http://jenkinsci.jenkinsci.svc.cluster.local

1.7、自定义jenkins

如果不使用提供的数据文件，自定义jenkins需要安装下列插件

  • Credentials
  • Credentials Binding Plugin
  • Git
  • Gitee
  • Pipeline
  • Git Parameter
  • Email Extension Template
  • Build With Parameters
  • Kubernetes
  • Kubernetes CLI

然后可参照文章：https://blog.csdn.net/gyfghh/article/details/130377218， 对所需内容进行配置

2、密钥配置

2.1、Gitlab

Dashboard->Manage Jenkins->Credentials->System->Global credentials->Username with password

在这里插入图片描述

2.2、Harbor

Dashboard->Manage Jenkins->Credentials->System->Global credentials->Username with password

在这里插入图片描述

2.3、kubernetes

Dashboard->Manage Jenkins->Credentials->System->Global credentials-> Secret file

文件选择kubernetes集群中的~/.kube/config

在这里插入图片描述

3、配置Agent

Dashboard->Manage Jenkins->Configure Global Security->Agents

将模式修改为Fixed，端口为50000
在这里插入图片描述

4、配置kubernetes集群

Dashboard->Manage Jenkins->Manage Nodes and Clouds->Configure Clouds

选择上面步骤创建的kubernetes凭证，随后测试连接
在这里插入图片描述

5、镜像准备

接下来需要在jenkins slave里面进行项目打包、编译、镜像构建、集群中部署，因此需要两个中间镜像：docker、kubectl

5.1、idocker

# 如不想从头制作，可使用registry.cn-hangzhou.aliyuncs.com/ialso/idocker:latest
# Build: docker build --no-cache --force-rm -t idocker ./
# Run: docker run -d --name idocker -v /var/run/docker.sock:/var/run/docker.sock idocker
# Into: docker exec -it idocker /bin/sh
FROM alpine:3.17

# 设置apk源
RUN echo "http://mirrors.tuna.tsinghua.edu.cn/alpine/v3.17/main" > /etc/apk/repositories && 
    echo "http://mirrors.tuna.tsinghua.edu.cn/alpine/v3.17/community" >> /etc/apk/repositories && 
    apk update && 
    apk add docker-cli

ENTRYPOINT ["/bin/sh", "-c", "sleep 600"]

5.2、ikubectl

# 如不想从头制作，可使用registry.cn-hangzhou.aliyuncs.com/ialso/ikubectl:latest
# Build: docker build --no-cache --force-rm -t ikubectl ./
# Run: docker run -d --name ikubectl ikubectl
# Into: docker exec -it ikubectl /bin/sh
FROM alpine:3.17

# 在https://github.com/kubernetes/kubernetes/tree/master/CHANGELOG下载自己集群对应版本的kubectl
COPY kubectl /usr/local/bin/kubectl

ENTRYPOINT ["/bin/sh", "-c", "sleep 600"]

6、配置slave

6.1、项目配置

需在项目中将Jenkinsfile配置为Pipeline script from SCM， 并配置相关信息

在这里插入图片描述

6.2、Jenkinsfile

此文件需放置在项目根目录

// Jenkinsfile
pipeline {
    // 配置代理
    agent {
        // kubernetes代理信息
        kubernetes {
            // 这里要填写Jenkins configureClouds中配置的kubernetes信息
            cloud 'kubernetes'
            label "jenkinsci"
            // 超时时间
            slaveConnectTimeout 1200
            // podTemplate
            yamlFile 'PodTemplate.yaml'
        }
    }
    environment {
        // 镜像名称
        IMAGE_NAME = "yanyang"
        // 获取commit id（short形式）作为镜像tag
        IMAGE_TAG = sh(returnStdout: true, script: 'git rev-parse --short HEAD').trim()

        // 镜像仓库地址
        HUB_ADDRESS = "registry.cn-hangzhou.aliyuncs.com"
        // 镜像仓库
        HUB_REGISTRY = "ialso"
        // 账户信息
        HUB_USER = credentials('aliyunHub')
    }
    stages {
        stage('Pull') {
            steps {
                checkout scmGit(branches: [[name: '*/master']], extensions: [], userRemoteConfigs: [[credentialsId: 'gitee', url: 'https://gitee.com/xumeng03/yanyang.git']])
            }
        }
        stage('Build && Push') {
            steps {
                container(name: 'docker'){
                    // 测试kubectl命令
                    sh """
                        docker images
                    """
                }
            }
        }
        stage('Deploy') {
            environment {
                KUBE_CONFIG = credentials('kubernetes')
            }
            steps {
                container(name: 'kuberctl'){
                    // 测试kubectl命令
                    sh """
                        kubectl --kubeconfig ${KUBE_CONFIG} get nodes
                    """
                }
            }
        }
    }
}

6.3、podTemplate

此文件需放置在yamlFile能对应到的位置，这里是项目根目录

apiVersion: v1
kind: Pod
metadata:
  name: jenkinsci
  namespace: devops
spec:
  containers:
    - name: jnlp
      image: jenkins/jnlp-slave:4.9-1-jdk11
      imagePullPolicy: IfNotPresent
      volumeMounts:
        - name: timezone
          mountPath: /etc/localtime
    - name: docker
      image: registry.cn-hangzhou.aliyuncs.com/ialso/idocker:latest
      imagePullPolicy: IfNotPresent
      command:
        - "cat"
      tty: true
      volumeMounts:
        - name: timezone
          mountPath: /etc/localtime
        - name: docker
          mountPath: /var/run/docker.sock
    - name: kuberctl
      image: registry.cn-hangzhou.aliyuncs.com/ialso/ikubectl:latest
      imagePullPolicy: IfNotPresent
      command:
        - "cat"
      tty: true
      volumeMounts:
        - name: timezone
          mountPath: /etc/localtime
  volumes:
    - name: timezone
      hostPath:
        path: /usr/share/zoneinfo/Asia/Shanghai
    - name: docker
      hostPath:
        path: /var/run/docker.sock

7、示例

7.0、镜像拉取密钥配置

注意secret具有命名空间隔离性！！！

kubectl create secret -n name-space 
	docker-registry <name> 
	--docker-server=DOCKER_REGISTRY_SERVER 
	--docker-username=DOCKER_USER 
	--docker-password=DOCKER_PASSWORD

7.1、Vue

目前更新至推送镜像阶段

PodTemplate.yaml

apiVersion: v1
kind: Pod
metadata:
  name: jenkinsci
  namespace: devops
spec:
  containers:
    - name: jnlp
      image: jenkins/jnlp-slave:4.9-1-jdk11
      imagePullPolicy: IfNotPresent
      volumeMounts:
        - name: timezone
          mountPath: /etc/localtime
    - name: docker
      image: registry.cn-hangzhou.aliyuncs.com/ialso/idocker:latest
      imagePullPolicy: IfNotPresent
      command:
        - "cat"
      tty: true
      volumeMounts:
        - name: timezone
          mountPath: /etc/localtime
        - name: docker
          mountPath: /var/run/docker.sock
    - name: kuberctl
      image: registry.cn-hangzhou.aliyuncs.com/ialso/ikubectl:latest
      imagePullPolicy: IfNotPresent
      command:
        - "cat"
      tty: true
      volumeMounts:
        - name: timezone
          mountPath: /etc/localtime
  volumes:
    - name: timezone
      hostPath:
        path: /usr/share/zoneinfo/Asia/Shanghai
    - name: docker
      hostPath:
        path: /var/run/docker.sock

deployment.tql

# DEPLOY_NAMESPACE：部署命名空间
# DEPLOY_NAME：部署名称
# APPLICATION_REPLICAS：应用部署份数
# APPLICATION_NAME：应用名称
# IMAGE_SECRET：镜像下载密钥
# HUB_ADDRESS：镜像仓库地址
# HUB_REGISTRY：镜像仓库
# IMAGE_NAME：镜像名称
# IMAGE_TAG：镜像标签
# APPLICATION_PORT：应用端口
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {DEPLOY_NAME}
  namespace: {DEPLOY_NAMESPACE}
  labels:
    app: {DEPLOY_NAME}
spec:
  replicas: {APPLICATION_REPLICAS}
  selector:
    matchLabels:
      app: {APPLICATION_NAME}
  template:
    metadata:
      name: {APPLICATION_NAME}
      labels:
        app: {APPLICATION_NAME}
    spec:
      imagePullSecrets:
        - name: {IMAGE_SECRET}
      containers:
        - name: {APPLICATION_NAME}
          image: {HUB_ADDRESS}/{HUB_REGISTRY}/{IMAGE_NAME}:{IMAGE_TAG}
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: {APPLICATION_PORT}
          volumeMounts:
            - name: timezone
              mountPath: /etc/localtime
      restartPolicy: Always
      volumes:
        # 时间处理
        - name: timezone
          hostPath:
            path: /usr/share/zoneinfo/Asia/Shanghai
---
apiVersion: v1
kind: Service
metadata:
  name: {APPLICATION_NAME}-service
  namespace: {DEPLOY_NAMESPACE}
spec:
  selector:
    app: {APPLICATION_NAME}
  ports:
    - protocol: TCP
      port: 80
      targetPort: {APPLICATION_PORT}
  type: ClusterIP
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: {APPLICATION_NAME}
  namespace: {DEPLOY_NAMESPACE}
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
    - host: ialso.cn
      http:
        paths:
          - pathType: Prefix
            path: /
            backend:
              service:
                name: {APPLICATION_NAME}-service
                port:
                  number: 80

Jenkinsfile

// Jenkinsfile
pipeline {
    // 配置代理
    agent {
        // kubernetes代理信息
        kubernetes {
            // 这里要填写Jenkins configureClouds中配置的kubernetes信息
            cloud 'kubernetes'
            label "jenkinsci"
            // 超时时间
            slaveConnectTimeout 1200
            // podTemplate
            yamlFile 'PodTemplate.yaml'
        }
    }
    environment {
        // 镜像名称
        IMAGE_NAME = "yanyang"
        // 获取commit id（short形式）作为镜像tag
        IMAGE_TAG = sh(returnStdout: true, script: 'git rev-parse --short HEAD').trim()

        // 镜像仓库地址
        HUB_ADDRESS = "registry.cn-hangzhou.aliyuncs.com"
        // 镜像仓库
        HUB_REGISTRY = "ialso"
        // 账户信息
        HUB_USER = credentials('aliyunHub')

        // 部署命名空间
        DEPLOY_NAMESPACE = "ialso"
        // 部署名称
        DEPLOY_NAME = "yanyang"
        // 应用部署份数
        APPLICATION_REPLICAS = 1
        // 应用名称
        APPLICATION_NAME = "yanyang"
        // 应用镜像下载密钥
        IMAGE_SECRET = "aliyun-hub"
        // 应用端口
        APPLICATION_PORT = 80
    }
    stages {
        stage('Pull') {
            steps {
                checkout scmGit(branches: [[name: '*/master']], extensions: [], userRemoteConfigs: [[credentialsId: 'gitee', url: 'https://gitee.com/xumeng03/yanyang.git']])
            }
        }
        stage('Build && Push') {
            steps {
                container(name: 'docker'){
                    sh """
                        docker build --no-cache -t ${HUB_ADDRESS}/${HUB_REGISTRY}/${IMAGE_NAME}:${IMAGE_TAG} ./
                        docker login -u ${HUB_USER_USR} -p ${HUB_USER_PSW} ${HUB_ADDRESS}
                        docker push ${HUB_ADDRESS}/${HUB_REGISTRY}/${IMAGE_NAME}:${IMAGE_TAG}
                    """
                }
            }
        }
        stage('Deploy') {
            environment {
                KUBE_CONFIG = credentials('kubernetes')
            }
            steps {
                container(name: 'kuberctl'){
                    sh """
                        sed -e 's#{DEPLOY_NAMESPACE}#${DEPLOY_NAMESPACE}#g;s#{DEPLOY_NAME}#${DEPLOY_NAME}#g;s#{APPLICATION_REPLICAS}#${APPLICATION_REPLICAS}#g;s#{APPLICATION_NAME}#${APPLICATION_NAME}#g;s#{IMAGE_SECRET}#${IMAGE_SECRET}#g;s#{HUB_ADDRESS}#${HUB_ADDRESS}#g;s#{HUB_REGISTRY}#${HUB_REGISTRY}#g;s#{IMAGE_NAME}#${IMAGE_NAME}#g;s#{IMAGE_TAG}#${IMAGE_TAG}#g;s#{APPLICATION_PORT}#${APPLICATION_PORT}#g;' deployment.tql > deployment.yaml
                        cat deployment.yaml
                        kubectl --kubeconfig ${KUBE_CONFIG} apply -f deployment.yaml
                    """
                }
            }
        }
    }
}

7.2、Java

风语者！平时喜欢研究各种技术，目前在从事后端开发工作，热爱生活、热爱工作。

站长推荐