Minikube – 配置 Jenkins Kubernetes plugin

在这里插入图片描述

文章目录

- 1. 配置 kubernetes credentials
- 2. 安装 kubernets plugin
- 3. 安装 docker 插件
- 4. 连接 minikube 集群
- 5. Pod template 参数
- 6. Container template 参数
- 7. 实例
- - 7.1 创建一个简单 pod
  - 7.2 pod name 变化
  - 7.3 指定 namespace
  - 7.4 volumes 挂载
  - 7.5 Liveness Probe 探针
  - 7.6 创建多个容器
  - 7.7 继承
  - 7.8 pod 嵌套
  - 7.9 Pipeline script from SCM
  - 7.10 复用 groovy 文件
  - 7.11 声明式 pipeline
- 8 应用实践
- - 8.1 git 拉取仓库 & 构建镜像
  - 8.2 编写 Dockerfile & 构建镜像
  - 8.1 kaniko 构建镜像 & 推送入库

1. 配置 kubernetes credentials

具体可官方添加 credentials 步骤

在这里插入图片描述

在这里插入图片描述

获取 token

 kubectl get secrets -n jenkins jenkins-token-6r26g -oyaml
apiVersion: v1
data:
  ca.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCakNDQWU2Z0F3SUJBZ0lCQVRBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwdGFXNXAKYTNWaVpVTkJNQjRYRFRJeU1URXlPVEEyTURJd09Gb1hEVE15TVRFeU56QTJNREl3T0Zvd0ZURVRNQkVHQTFVRQpBeE1LYldsdWFXdDFZbVZEUVRDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTU8xCkpIMER5VU5DeGh2dHdtR05EaGxORTZFTXJWMzl0N1dmUjJmMTJEM3U4SVlpckFhdFBNb1RYZThpTDR4NXl5ckoKaUhRdWtIMXAzcjhqS0E4WVpaa2cwN3FIOE1mYXB2dG9qQjQ3QUROanBLYUNVcXh6UFlvY3l0VlU3UDA0dDhMVQpyRTZFTG9qcGlWcWNEdzZSakhEQ3p2R3NFU1NvTUIyZTlxVXN2dU9kMmdlMFVwMnJKdjRmTTY3aEdJN3FIYkJ3CjVFbE5tMzcySytxOS9nUFBtSW1kQU94Y2xFOENTcy9aYWxuV1AzcWdLbGVoNnQxbkFscnhpbXpaSVdkYnZvMzYKemxTRmwwaHlrZGNWL0RUdndaNFYvMkR1OVVtbWpsWFJ5cWdOYXF1R2lTYXVMRjhWRkpYWnROWmhSR2NWZW1tNQpRQ0gyWWFEN3lzMmJOYlM3YUtzQ0F3RUFBYU5oTUY4d0RnWURWUjBQQVFIL0JBUURBZ0trTUIwR0ExVWRKUVFXCk1CUUdDQ3NHQVFVRkJ3TUNCZ2dyQmdFRkJRY0RBVEFQQmdOVkhSTUJBZjhFQlRBREFRSC9NQjBHQTFVZERnUVcKQkJSVXg5bTFqVlIrR1llNXJMalZhRmpxaXljbjZEQU5CZ2txaGtpRzl3MEJBUXNGQUFPQ0FRRUF3M3JpRkY1QgozTXZSZmFlRnB0d3pRRzYzVEZncVoxN3luaDBSV2xrUkM3M3c0L1BqczhUUmhXRFNYTy94elNROTNJNjNlaHkwCldIOTJsMDJhN3NNTXVvY1YrRG8vTUlpVmI3bVA2VmtKbjF2Vm43cE4zM2c5dDJFWkY2Yis0Q0JUUUo5YXlodGwKMHo5Y3hwMXVaMEt4SUZ1bzJuY3lPREFEZ293T3h0WHJKZ1h5ckE5MldFbi91NUl5VnJ1Q2ZlWjk0YTMwN3NVUAplMXpnTlBBam8xUHdTQVY4bE5tRkpZbG9FVzNhRkE1MDU3STZYZy81dEg0QzJ2c3NkNVFxS2R4RHM1UU5QK2VpClhZblp2S2dTTmJSVUtyL1NIM1Q5NkQ0azIrQXpsWEhBYVRwYk9uQURzYmt3YTYwcHUvWmJsa0dLajhFVE9KL2UKRldleGYyUmtzbkVWN1E9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
  namespace: amVua2lucw==
  token: ZXlKaGJHY2lPaUpTVXpJMU5pSXNJbXRwWkNJNkltdHpSM2xwUTJrd2FUUmxiR3gyYkMwMFIyVndOR0ZUZEZsVlVXUTRVaTFNUTBnMU9ISlBiRkJOWldNaWZRLmV5SnBjM01pT2lKcmRXSmxjbTVsZEdWekwzTmxjblpwWTJWaFkyTnZkVzUwSWl3aWEzVmlaWEp1WlhSbGN5NXBieTl6WlhKMmFXTmxZV05qYjNWdWRDOXVZVzFsYzNCaFkyVWlPaUpxWlc1cmFXNXpJaXdpYTNWaVpYSnVaWFJsY3k1cGJ5OXpaWEoyYVdObFlXTmpiM1Z1ZEM5elpXTnlaWFF1Ym1GdFpTSTZJbXBsYm10cGJuTXRkRzlyWlc0dE5uSXlObWNpTENKcmRXSmxjbTVsZEdWekxtbHZMM05sY25acFkyVmhZMk52ZFc1MEwzTmxjblpwWTJVdFlXTmpiM1Z1ZEM1dVlXMWxJam9pYW1WdWEybHVjeUlzSW10MVltVnlibVYwWlhNdWFXOHZjMlZ5ZG1salpXRmpZMjkxYm5RdmMyVnlkbWxqWlMxaFkyTnZkVzUwTG5WcFpDSTZJak01TnpjMU0yUmtMVEJtTTJZdE5ERmxNaTFoTnpOakxUaGhOVEV6TldRMlptRmhZU0lzSW5OMVlpSTZJbk41YzNSbGJUcHpaWEoyYVdObFlXTmpiM1Z1ZERwcVpXNXJhVzV6T21wbGJtdHBibk1pZlEuamx6SjV1bmRlZ3ZUMzVmVmViY3JsWnUxa09talAwTGpLenhsb2sxTHhINVowcm9jTmVsSmVJMXM4NWtnT1JXZGNuanFaTlg3VHpxano1eml1OS1peGZuY1UtTVRicTlRcTVMcnMyd1c1cGRTZTFwSlpWcmx5X2pGN2tOdXFSWENZZkdWVUFwbGFELXNES2ZJdFhmdEJ2dHpvWTgzV0QxdEdZMkJkSzRKM3NGdDRIdmw2YUVuUlZFejJ5WlRBTVBKZEJ1NUI4NjJCUnFOMEFicG13UFByM3FtellaZGZVQzNzdlBIVE9BWGxxUUpUcHRQbDVmMS05dEtKZUlGUGNLY1F5WTZSc0RXbUFfRW1sTHV6MWJWRkpxV2pxWnVqa3cyTndYeXhvS1VMNjBhVm9Lc2dQTVFpTU44TEg4Z2s0bTg5STQ5VjVvb2NpX3N3VTNYVy05cjNR
kind: Secret
metadata:
  annotations:
    kubernetes.io/service-account.name: jenkins
    kubernetes.io/service-account.uid: 397753dd-0f3f-41e2-a73c-8a5135d6faaa
  creationTimestamp: "2022-12-03T13:49:22Z"
  name: jenkins-token-6r26g
  namespace: jenkins
  resourceVersion: "199550"
  uid: 0b896aba-1810-436a-9eb8-dd0502e9e79f
type: kubernetes.io/service-account-token

复制token内容至secret
在这里插入图片描述

2. 安装 kubernets plugin

在这里插入图片描述

3. 安装 docker 插件

在这里插入图片描述

4. 连接 minikube 集群

在这里插入图片描述

name: 定义集群名称
kubernetes credentials 选择下拉刚刚创建的 minikube2
kubernetes namespace 选择自己创建的 namespace。
Kubernetes URL 与 Kubernetes server certificate key 通过命令 kubectl config view获取
填完点击测试一下，再保存。

$ kubectl  config view
apiVersion: v1
clusters:
- cluster:
    certificate-authority: /root/.minikube/ca.crt
    extensions:
    - extension:
        last-update: Wed, 30 Nov 2022 14:20:59 CST
        provider: minikube.sigs.k8s.io
        version: v1.28.0
      name: cluster_info
    server: https://192.168.10.26:8443
  name: minikube
contexts:
- context:
    cluster: minikube
    extensions:
    - extension:
        last-update: Wed, 30 Nov 2022 14:20:59 CST
        provider: minikube.sigs.k8s.io
        version: v1.28.0
      name: context_info
    namespace: default
    user: minikube
  name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
  user:
    client-certificate: /root/.minikube/profiles/minikube/client.crt
    client-key: /root/.minikube/profiles/minikube/client.key

5. Pod template 参数

cloud ：Jenkins设置中定义的云的名称。默认为kubernetes
name： pod名称.
namespace ：pod 命名空间.
label ：节点标签。这就是在通过节点步骤请求代理时可以引用 pod 模板的方式。在管道中，建议省略此字段并依赖生成的标签，该标签可以使用 podTemplate 块中定义的 POD_LABEL 变量引用。
yaml Pod 的 yaml 表示，以允许设置任何不支持的值作为字段
yamlMergeStrategy ：merge() or override(). 控制 yaml 定义是否覆盖或与从使用 inheritFrom 声明的 pod 模板继承的 yaml 定义合并。默认为 override() （出于向后兼容性原因）。
containers： container templates 部分
serviceAccount ：pod 服务帐户.
nodeSelector ：pod的节点选择器.
nodeUsageMode：要么NORMAL要么EXCLUSIVE，该参数控制的是只调度标签表达式匹配的作业还是尽可能使用节点.
volumes：挂载持久存储卷
- configMapVolume : 挂载ConfigMap，只读.
- dynamicPVC() : 动态管理的持久卷 pvc ，它与 pod 同时被删除.
- emptyDirVolume (default): 空目录
- hostPathVolume() : 挂载主机目录
- nfsVolume() : 挂载NFS目录
- persistentVolumeClaim() : 绑定pvc.
- secretVolume : 挂载 secret，只读，适用于证书、用户密码.
envVars：应用于所有容器的环境变量.
- envVar：个环境变量，其值是内联定义的.
- secretEnvVar：通过secret对象定义一个环境变量.
imagePullSecrets：提取 secret 名称, 从私有 Docker 仓库中提取镜像.
annotations：pod注释
inheritFrom：要继承的一个或多个 pod 模板的列表
slaveConnectTimeout ：agent 在线超时秒数
podRetention：控制保留代理 pod 的行为。可以是 ‘never()’、'onFailure()'、‘always()’ 或 ‘default()’ - 如果为空，将默认在 activeDeadlineSeconds 过后删除 pod。
activeDeadlineSeconds：如果 podRetention 设置为 never() 或 onFailure()，则 pod 将在截止日期过后删除。
idleMinutes：允许 pod 保持活动状态以供重用，直到自上一个步骤执行后配置的分钟数过去。
showRawYaml：启用或禁用原始pod清单（manifest）的输出。默认为true
runAsUser：定义运行pod中所有容器的用户ID.
runAsGroup：定义在pod中运行所有容器的组ID.
hostNetwork：主机网络.
workspaceVolume：定义workspace卷的类型.
- dynamicPVC() ：挂载动态管理的pvc，会与pod一同被删除.
- emptyDirWorkspaceVolume (default): 定义主机上分配的空目录
- hostPathWorkspaceVolume() : 挂载主机目录
- nfsWorkspaceVolume() : 挂载nfs volume
- persistentVolumeClaimWorkspaceVolume() : 绑定pvc.

6. Container template 参数

Container template可通过用户界面与pipeline配置。

name：容器名字.
image ：镜像名称.
envVars ：应用于容器的环境变量(补充和覆盖在pod已设置的环境变量).
- envVar：一个环境变量，其值是内联定义的.
- secretEnvVar ：通过secret对象定义一个环境变量…
command：容器将执行的命令。将覆盖 Docker 入口点。常用命令：sleep.
args ：传递给命令的参数。例如：99999999 .
ttyEnabled：标志，以标记tty应该启用.
livenessProbe ：探针（不支持 - httpGet liveness probes）
ports：暴露容器上的端口
alwaysPullImage：容器将在启动时拉取镜像.
runAsUser：定义运行容器的用户ID.
runAsGroup：定义运行容器的组ID.

7. 实例

7.1 创建一个简单 pod

在这里插入图片描述

创建一个包含 docker 容器的 pod ，并且带有build标签。创建完后，使用docker version命令执行查看容器内docker 的版本信息。

Pipeline script：

podTemplate(label: 'build', containers: [
    containerTemplate(name: 'docker', image: 'docker', command: 'cat', ttyEnabled: true)
  ],
  volumes: [
    hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock'),
  ]
  ) {
    node('build') {
      container('docker') {
        sh 'docker version'
      }        
    }  
  }

点击执行
在这里插入图片描述
构建成功。

$ k get pods -n jenkins
NAME                READY   STATUS    RESTARTS   AGE
build-cb6cf-twpw7   2/2     Running   0          6s

7.2 pod name 变化

创建pod，编写podTemplate，通过yaml 定义 pod内容。

podTemplate(yaml: '''
    apiVersion: v1
    kind: Pod
    metadata:
      labels: 
        some-label: some-label-value
    spec:
      containers:
      - name: busybox
        image: busybox
        command:
        - sleep
        args:
        - 99d
    ''') {
    node(POD_LABEL) {
      container('busybox') {
        echo POD_CONTAINER // displays 'busybox'
        sh 'hostname'
      }
    }
}

第一次构建部署输出：

$ k get pods -n jenkins | grep busybox
busybox-1-zdnpq-mwqtd-0vfg0   2/2     Running   0          19s

busybox pod前缀默认是项目名字
1 代表第一次构建
zdnpq-mwqtd-0vfg0是随机码

当我们第二次构建，pod 名字会发生变化，保持pod名字的唯一性。

$  k get pods -n jenkins | grep busybox
busybox-2-c7blp-840k1-s5c5d   2/2     Running   0          20s

我们尝试修改项目名称为busybox2
在这里插入图片描述
输出结果

$ k get pods -n jenkins | grep busybox
busybox2-3-2fc4l-8z8dp-2t0c7   2/2     Running   0          7s

原来的busybox变成busybox2，并且表明第三次构建。

我们尝试添加定义pod的name为busybox3

podTemplate(name: "busybox3",yaml: '''
    apiVersion: v1
    kind: Pod
    metadata:
      labels: 
        some-label: some-label-value
    spec:
      containers:
      - name: busybox
        image: busybox
        command:
        - sleep
        args:
        - 99d
    ''') {
    node(POD_LABEL) {
      container('busybox') {
        echo POD_CONTAINER // displays 'busybox'
        sh 'hostname'
      }
    }
}

当我们自定义pod 的name，构建次数会取消，项目名字不再是pod的前缀。

k get pods -n jenkins | grep busybox
busybox3-lzhfl-7hflx   2/2     Running   0          20s

7.3 指定 namespace

然后，我们尝试换一个已创建好的名字为one的namespace进行构建。

podTemplate(name: "busybox4",
    namespace: "one",
    yaml: '''
    ....

$ k get pods -n one  | grep busybox
busybox4-fxd38-5vpwf   2/2     Running   0          24s

7.4 volumes 挂载

挂载本地 /var/run/docker.sock 文件

podTemplate(name: "busybox5",
    namespace: "one",
    yaml: '''
    apiVersion: v1
    kind: Pod
    metadata:
      labels: 
        some-label: some-label-value
    spec:
      containers:
      - name: busybox
        image: busybox
        command:
        - sleep
        args:
        - 99d
    ''',
      volumes: [
    hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock'),
  ]
    ) {
    node(POD_LABEL) {
      container('busybox') {
        echo POD_CONTAINER // displays 'busybox'
        sh 'hostname'
      }
    }
}

7.5 Liveness Probe 探针

podTemplate(name: "busybox6",namespace: "one",
containers: [
containerTemplate(name: 'busybox', image: 'busybox', command: 'sleep', args: '99d',
                  livenessProbe: containerLivenessProbe(execArgs: 'ping 127.0.0.1', initialDelaySeconds: 30, timeoutSeconds: 1, failureThreshold: 3, periodSeconds: 10, successThreshold: 1)
)]
){
    node(POD_LABEL) {
      container('busybox') {
        echo POD_CONTAINER // displays 'busybox'
        sh 'hostname'
      }
    }
}

更多kubernetes探针内容：Configure Liveness, Readiness and Startup Probes

7.6 创建多个容器

podTemplate(name: "build1",namespace: "one",containers: [
    containerTemplate(name: 'maven', image: 'maven:3.8.1-jdk-8', command: 'sleep', args: '99d'),
    containerTemplate(name: 'golang', image: 'golang:1.16.5', command: 'sleep', args: '99d')
  ]) {

    node(POD_LABEL) {
        stage('Get a Maven project') {
            git 'https://github.com/jenkinsci/kubernetes-plugin.git'
            container('maven') {
                stage('Build a Maven project') {
                    sh 'mvn -B -ntp clean install'
                }
            }
        }

        stage('Get a Golang project') {
            git url: 'https://github.com/hashicorp/terraform.git', branch: 'main'
            container('golang') {
                stage('Build a Go project') {
                    sh '''
                    mkdir -p /go/src/github.com/hashicorp
                    ln -s `pwd` /go/src/github.com/hashicorp/terraform
                    cd /go/src/github.com/hashicorp/terraform && make
                    '''
                }
            }
        }

    }
}

查看创建的 pod

$ kubectl get pods -n one
NAME                 READY   STATUS    RESTARTS   AGE
build1-qc5pr-679mv   3/3     Running   0          17m
#三个容器
$ kubectl get pods -n one -o jsonpath="{.items[*].spec.containers[*].name}" | tr -s '[[:space:]]' '\n' |sort |uniq
golang
jnlp
maven

注意：jnlp容器是Jenkins代理,并且命令默认将在运行Jenkins代理的jnlp容器中执行。(jnlp的名称是历史的，为了兼容而保留。)，例如：

podTemplate {
    node(POD_LABEL) {
        stage('Run shell') {
            sh 'echo hello world'
        }
    }
}

我们还可以通过yaml定义格式：

podTemplate(name: "build1",namespace: "one",yaml: '''
    apiVersion: v1
    kind: Pod
    spec:
      containers:
      - name: maven
        image: maven:3.8.1-jdk-8
        command:
        - sleep
        args:
        - 99d
      - name: golang
        image: golang:1.16.5
        command:
        - sleep
        args:
        - 99d
''') {
  node(POD_LABEL) {
    stage('Get a Maven project') {
      git 'https://github.com/jenkinsci/kubernetes-plugin.git'
      container('maven') {
        stage('Build a Maven project') {
          sh 'mvn -B -ntp clean install'
        }
      }
    }

    stage('Get a Golang project') {
      git url: 'https://github.com/hashicorp/terraform-provider-google.git', branch: 'main'
      container('golang') {
        stage('Build a Go project') {
          sh '''
            mkdir -p /go/src/github.com/hashicorp
            ln -s `pwd` /go/src/github.com/hashicorp/terraform
            cd /go/src/github.com/hashicorp/terraform && make
          '''
        }
      }
    }

  }
}

7.7 继承

pod template 可以继承现有模板，也可以不继承。这意味着pod template将从它所继承的模板继承node selector, service account, image pull secrets, container templates and volumes。

yaml根据yamlMergeStrategy的值进行合并。

我通过界面创建 pod template
在这里插入图片描述

在这里插入图片描述

在项目编写pipeline script：

podTemplate(name: "build2",namespace: "one",inheritFrom: 'mypod', containers: [
    containerTemplate(name: 'maven', image: 'maven:3.8.1-jdk-11')
  ]) {
  node(POD_LABEL) {
    stage('Get a Maven project') {
      git 'https://github.com/jenkinsci/kubernetes-plugin.git'
      container('maven') {
        stage('Build a Maven project') {
          sh 'mvn -B -ntp clean install'
        }
      }
    }

    stage('Get a Golang project') {
      git url: 'https://github.com/hashicorp/terraform-provider-google.git', branch: 'main'
      container('golang') {
        stage('Build a Go project') {
          sh '''
            mkdir -p /go/src/github.com/hashicorp
            ln -s `pwd` /go/src/github.com/hashicorp/terraform
            cd /go/src/github.com/hashicorp/terraform && make
          '''
        }
      }
    }
  }
}

$ kubectl get pods -n one
NAME                 READY   STATUS    RESTARTS   AGE
build2-4p074-b1skx   3/3     Running   0          11m

$ kubectl get pods build2-4p074-b1skx -n one -o jsonpath="{.spec.containers[*].image}" | tr -s '[[:spa
ce:]]' '\n' |sort |uniq
golang:1.16.5
jenkins/inbound-agent:4.11-1-jdk11
maven:3.8.1-jdk-11

这里看到：通过inheritFrom: 'mypod'实现对模板的继承，并且实现mypod模板中的maven版本v1.8.1-jdk-8升级成为3.8.1-jdk-11，完成覆盖。

7.8 pod 嵌套

两个 pod template 合成一个包含两个容器的 pod实例。

podTemplate(name: 'docker', namespace: 'one', containers: [containerTemplate(image: 'docker', name: 'docker', command: 'cat', ttyEnabled: true)]) {
    podTemplate(name: 'maven', namespace: 'one', containers: [containerTemplate(image: 'maven:3.8.1-jdk-11', name: 'maven', command: 'cat', ttyEnabled: true)]) {
    node(POD_LABEL) {
      container('docker') {
        sh "echo hello from $POD_CONTAINER" // displays 'hello from docker'
      }
      container('maven') {
        sh "echo hello from $POD_CONTAINER" // displays 'hello from maven'
      }
     }
    }
}

终端：

$ k get pods -n one
NAME                READY   STATUS    RESTARTS   AGE
maven-9h33d-t39gn   3/3     Running   0          5s

#显示pod容器名称
$ kubectl get pods maven-9h33d-t39gn -n one -o jsonpath="{.spec.containers[*].name}" | tr -s '[[:space:]]' '\n' | sort |uniq
docker
jnlp
maven

console output 输出：

Started by user Jenkins Admin
[Pipeline] Start of Pipeline
[Pipeline] podTemplate
[Pipeline] {
[Pipeline] podTemplate
[Pipeline] {
[Pipeline] node
Created Pod: minikube2 one/maven-1bmzz-wv1sp
Agent maven-1bmzz-wv1sp is provisioned from template maven-1bmzz
---
apiVersion: "v1"
kind: "Pod"
metadata:
  annotations:
    buildUrl: "http://jenkins.jenkins.svc.cluster.local:8080/job/busybox2/30/"
    runUrl: "job/busybox2/30/"
  labels:
    jenkins/jenkins-jenkins-agent: "true"
    jenkins/label-digest: "4ba5a4e248fa7baa06e229917703aa75dbced0ac"
    jenkins/label: "busybox2_30-lt9wl"
  name: "maven-1bmzz-wv1sp"
  namespace: "one"
spec:
  containers:
  - command:
    - "cat"
    image: "maven:3.8.1-jdk-11"
    imagePullPolicy: "IfNotPresent"
    name: "maven"
    resources:
      limits: {}
      requests: {}
    tty: true
    volumeMounts:
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
      readOnly: false
  - command:
    - "cat"
    image: "docker"
    imagePullPolicy: "IfNotPresent"
    name: "docker"
    resources:
      limits: {}
      requests: {}
    tty: true
    volumeMounts:
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
      readOnly: false
  - env:
    - name: "JENKINS_SECRET"
      value: "********"
    - name: "JENKINS_TUNNEL"
      value: "jenkins-agent.jenkins.svc.cluster.local:50000"
    - name: "JENKINS_AGENT_NAME"
      value: "maven-1bmzz-wv1sp"
    - name: "JENKINS_NAME"
      value: "maven-1bmzz-wv1sp"
    - name: "JENKINS_AGENT_WORKDIR"
      value: "/home/jenkins/agent"
    - name: "JENKINS_URL"
      value: "http://jenkins.jenkins.svc.cluster.local:8080/"
    image: "jenkins/inbound-agent:4.11-1-jdk11"
    name: "jnlp"
    resources:
      limits: {}
      requests:
        memory: "256Mi"
        cpu: "100m"
    volumeMounts:
    - mountPath: "/home/jenkins/agent"
      name: "workspace-volume"
      readOnly: false
  nodeSelector:
    kubernetes.io/os: "linux"
  restartPolicy: "Never"
  volumes:
  - emptyDir:
      medium: ""
    name: "workspace-volume"

Running on maven-1bmzz-wv1sp in /home/jenkins/agent/workspace/busybox2
[Pipeline] {
[Pipeline] container
[Pipeline] {
[Pipeline] sh
+ echo hello from docker
hello from docker
[Pipeline] }
[Pipeline] // container
[Pipeline] container
[Pipeline] {
[Pipeline] sh
+ echo hello from maven
hello from maven
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
Finished: SUCCESS

7.9 Pipeline script from SCM

在这里插入图片描述

在minikube helm 部署 jenkins中，pod jenkins-0内 git clone 仓库时遇到 gnutls_handshake() failed: Error in the pull function.。可能需要配置proxy或者升级 git,但也不排除 Jenkins 配置 Github ssh Credentials 的问题。我暂时本地部署Jenkins 体验Pipeline script from SCM相关实践。

kubernetes-plugin/examples/containerLog.groovy

podTemplate(yaml: '''
              apiVersion: v1
              kind: Pod
              metadata:
                labels:
                  some-label: some-label-value
              spec:
                containers:
                - name: maven
                  image: maven:3.8.1-jdk-8
                  command:
                  - sleep
                  args:
                  - 99d
                  tty: true
                - name: mongo
                  image: mongo
''') {
  node(POD_LABEL) {
    stage('Integration Test') {
      try {
        container('maven') {
          sh 'nc -z localhost:27017 && echo "connected to mongo db"'
          // sh 'mvn -B clean failsafe:integration-test' // real integration test

          def mongoLog = containerLog(name: 'mongo', returnLog: true, tailingLines: 5, sinceSeconds: 20, limitBytes: 50000)
          assert mongoLog.contains('connection accepted from 127.0.0.1:')
          sh 'echo failing build; false'
        }
      } catch (Exception e) {
        containerLog 'mongo'
        throw e
      }
    }
  }
}

7.10 复用 groovy 文件

Extending with Shared Libraries

package com.foo.utils

public void dockerTemplate(body) {
  podTemplate(
        containers: [containerTemplate(name: 'docker', image: 'docker', command: 'sleep', args: '99d')],
        volumes: [hostPathVolume(hostPath: '/var/run/docker.sock', mountPath: '/var/run/docker.sock')]) {
    body.call()
}
}

public void mavenTemplate(body) {
  podTemplate(
        containers: [containerTemplate(name: 'maven', image: 'maven', command: 'sleep', args: '99d')],
        volumes: [secretVolume(secretName: 'maven-settings', mountPath: '/root/.m2'),
                  persistentVolumeClaim(claimName: 'maven-local-repo', mountPath: '/root/.m2repo')]) {
    body.call()
}
}

注意，POD_LABEL将是生成的最里面的标签，以获得节点上所有外部pod都可用的节点，如本例所示

import com.foo.utils.PodTemplates

podTemplates = new PodTemplates()

podTemplates.dockerTemplate {
  podTemplates.mavenTemplate {
    node(POD_LABEL) {
      container('docker') {
        sh "echo hello from $POD_CONTAINER" // displays 'hello from docker'
      }
      container('maven') {
        sh "echo hello from $POD_CONTAINER" // displays 'hello from maven'
      }
     }
  }
}

在pipeline script 中，有时不需要这种通过嵌套声明的隐式继承，或者首选另一种显式继承。在这种情况下，使用inheritFrom 来删除任何继承，或者使用inheritFrom 'otherParent'来覆盖它。

7.11 声明式 pipeline

pipeline {
  agent {
    kubernetes {
      yaml '''
        apiVersion: v1
        kind: Pod
        metadata:
          labels:
            some-label: some-label-value
        spec:
          containers:
          - name: maven
            image: maven:alpine
            command:
            - cat
            tty: true
          - name: busybox
            image: busybox
            command:
            - cat
            tty: true
        '''
    }
  }
  stages {
    stage('Run maven') {
      steps {
        container('maven') {
          sh 'mvn -version'
        }
        container('busybox') {
          sh '/bin/busybox'
        }
      }
    }
  }
}

8 应用实践

8.1 git 拉取仓库 & 构建镜像

podTemplate(yaml: '''
              apiVersion: v1
              kind: Pod
              spec:
                containers:
                - name: docker
                  image: docker:19.03.1
                  command:
                  - sleep
                  args:
                  - 99d
                  volumeMounts:
                  - name: dockersock
                    mountPath: /var/run/docker.sock
                volumes:
                - name: dockersock
                  hostPath:
                    path: /var/run/docker.sock
''') {
  node(POD_LABEL) {
    stage('Build Docker image') {
      git 'https://github.com/jenkinsci/docker-inbound-agent.git'
      container('docker') {
        sh 'docker build -t jenkins/inbound-agent .'
      }
    }
  }
}

8.2 编写 Dockerfile & 构建镜像

podTemplate(yaml: '''
              apiVersion: v1
              kind: Pod
              spec:
                volumes:
                - name: docker-socket
                  emptyDir: {}
                containers:
                - name: docker
                  image: docker:19.03.1
                  command:
                  - sleep
                  args:
                  - 99d
                  volumeMounts:
                  - name: docker-socket
                    mountPath: /var/run
                - name: docker-daemon
                  image: docker:19.03.1-dind
                  securityContext:
                    privileged: true
                  volumeMounts:
                  - name: docker-socket
                    mountPath: /var/run
''') {
  node(POD_LABEL) {
    writeFile file: 'Dockerfile', text: 'FROM scratch'
    container('docker') {
      sh 'docker version && DOCKER_BUILDKIT=1 docker build --progress plain -t testing .'
    }
  }
}

8.1 kaniko 构建镜像 & 推送入库

podTemplate(yaml: '''
              kind: Pod
              spec:
                containers:
                - name: kaniko
                  image: gcr.io/kaniko-project/executor:v1.6.0-debug
                  imagePullPolicy: Always
                  command:
                  - sleep
                  args:
                  - 99d
                  volumeMounts:
                    - name: jenkins-docker-cfg
                      mountPath: /kaniko/.docker
                volumes:
                - name: jenkins-docker-cfg
                  projected:
                    sources:
                    - secret:
                        name: regcred
                        items:
                          - key: .dockerconfigjson
                            path: config.json
'''
  ) {

  node(POD_LABEL) {
    stage('Build with Kaniko') {
      git 'https://github.com/jenkinsci/docker-inbound-agent.git'
      container('kaniko') {
        sh '/kaniko/executor -f `pwd`/Dockerfile -c `pwd` --insecure --skip-tls-verify --cache=true --destination=mydockerregistry:5000/myorg/myimage'
      }
    }
  }
}

参考：

Kubernetes plugin for Jenkins
jenkinsci/kubernetes-plugin
jenkins pipeline 加载 groovy 脚本
Build & Push Docker Image using Jenkins Pipeline | Devops Integration Live Example Step By Step
Jenkins On Kubernetes Tutorial | How to setup Jenkins on kubernetes cluster | Thetips4you