aws codesuit 在codebuild和codepipeline中集成jenkins

codebuild集成jenkins

https://docs.aws.amazon.com/zh_cn/codebuild/latest/userguide/jenkins-plugin.html

Setting up a CI/CD pipeline by integrating Jenkins with AWS CodeBuild and AWS CodeDeploy

source选择本地的gitlab仓库，创建一个简单的springboot项目

在这里插入图片描述

在jenkins中下载安装codebuild和codepipeline插件

https://github.com/jenkinsci?q=aws-code&type=all&language=&sort=

在这里插入图片描述

源配置为本地的gitlab

在这里插入图片描述

在Build Steps中增加aws codebuild的步骤（需要配置aksk），注意由于创建codebuild项目的时候选择无source，所以需要欸之jenkins作为source。jenkins插件需要将gitlab仓库上传到s3中，然后交由codebuild构建。
在这里插入图片描述

直接构建，在构建中能看到具体的codebuild项目信息

在这里插入图片描述

在codebuild控制台查看构建，提交者是jenkins插件，并且虽然项目为nosource，但是实际上sourceversion仍旧能指向s3桶

在这里插入图片描述

相关错误

codebuild插件会实时将codebuild项目的构建日志回传到jenkins的控制台输出中

（1）没有凭证

需要为jenkins插件配置aksk凭证

[AWS CodeBuild Plugin] Authorization error
	> Invalid credentials ID. Verify that the credentials are of type CodeBuildCredentials and are accessible in this project. (Service: null; Status Code: 0; Error Code: null; Request ID: null; Proxy: null)
Build step 'AWS CodeBuild' changed build result to FAILURE

（2）没有项目

如果没有codebuild项目直接构建，jenkins找不到codebuild项目则报错

[AWS CodeBuild Plugin] Starting build with 
	> project name: helloBoot
	> Download build artifacts: false
[AWS CodeBuild Plugin] Error when calling CodeBuild StartBuild: 
	> Project cannot be found: arn:aws-cn:codebuild:cn-north-1:xxxxxxxxx:project/helloBoot (Service: AWSCodeBuild; Status Code: 400; Error Code: ResourceNotFoundException; Request ID: a5627552-9ab3-48ff-8431-a9a3f8ce1b19; Proxy: )
Build step 'AWS CodeBuild' changed build result to FAILURE

（3）没有pom文件

配置了projectsource默认使用codebuild的source，但是codebuild项目创建时配置了nosource，因此没有任何代码，因此也找不到pom文件。需要在构建项目中配置为jenkins源

[AWS CodeBuild Plugin] 2023/01/19 10:59:44 Running command mvn test && mvn package
[AWS CodeBuild Plugin] [INFO] Scanning for projects...
[AWS CodeBuild Plugin] [INFO] ------------------------------------------------------------------------
[AWS CodeBuild Plugin] [INFO] BUILD FAILURE
[AWS CodeBuild Plugin] [INFO] ------------------------------------------------------------------------
[AWS CodeBuild Plugin] [INFO] Total time:  0.086 s
[AWS CodeBuild Plugin] [INFO] Finished at: 2023-01-19T10:59:51Z
[AWS CodeBuild Plugin] [INFO] ------------------------------------------------------------------------
[AWS CodeBuild Plugin] [ERROR] The goal you specified requires a project to execute but there is no POM in this directory

（4）项目状态有误

选择jenkins项目后必须将source配置为s3，且指定s3桶名和路径

[AWS CodeBuild Plugin] CodeBuild configured improperly in project settings
	> When using Jenkins source, sourceTypeOverride must be S3 and you must specify both sourceTypeOverride and sourceLocationOverride
Build step 'AWS CodeBuild' changed build result to FAILURE

（5）没有权限

由于jenkins插件需要将源码上传到s3中，codebuild才能从s3获取到，如果codebuild角色无法访问s3，则会出先access denied的错误

[AWS CodeBuild Plugin] AccessDenied: Access Denied
[AWS CodeBuild Plugin] 	status code: 403, request id: 2960Q91F2RMZKG0W, host id: QBumXMzSyqV75541Ad0kqbBRga9Tzl7Ge5K/DsJ2OoFiOi9DDFExdIByxpRdZU+ZBk2knMNQQcM= for primary source and source version 19pEIsR9kR7fDGMSvYKv6LDauF8Rndsx
[AWS CodeBuild Plugin] Build helloBoot:56dfba8f-d367-450d-95a9-c9b24e5a6816 failed
	> AccessDenied: Access Denied	status code: 403, request id: 2960Q91F2RMZKG0W, host id: QBumXMzSyqV75541Ad0kqbBRga9Tzl7Ge5K/DsJ2OoFiOi9DDFExdIByxpRdZU+ZBk2knMNQQcM= for primary source and source version 19pEIsR9kR7fDGMSvYKv6LDauF8Rndsx (status code: CLIENT_ERROR)
Build step 'AWS CodeBuild' changed build result to FAILURE

如果使用流水线脚本，可使用片段生成器生成如下配置

awsCodeBuild artifactEncryptionDisabledOverride: '', artifactLocationOverride: '', artifactNameOverride: '', artifactNamespaceOverride: '', artifactPackagingOverride: '', artifactPathOverride: '', artifactTypeOverride: '', awsAccessKey: 'xxxxxxxx', awsSecretKey: 'yo7Lxxxxxxxxxxxxxxa7JU', buildSpecFile: '', buildTimeoutOverride: '', cacheLocationOverride: '', cacheModesOverride: '', cacheTypeOverride: '', certificateOverride: '', cloudWatchLogsGroupNameOverride: '', cloudWatchLogsStatusOverride: '', cloudWatchLogsStreamNameOverride: '', computeTypeOverride: '', credentialsId: '', credentialsType: 'keys', cwlStreamingDisabled: '', downloadArtifacts: 'false', downloadArtifactsRelativePath: '', envParameters: '', envVariables: '', environmentTypeOverride: '', exceptionFailureMode: '', gitCloneDepthOverride: '', imageOverride: '', insecureSslOverride: '', localSourcePath: '', overrideArtifactName: '', privilegedModeOverride: '', projectName: 'helloBoot', proxyHost: '', proxyPort: '', region: 'cn-north-1', reportBuildStatusOverride: '', s3LogsEncryptionDisabledOverride: '', s3LogsLocationOverride: '', s3LogsStatusOverride: '', secondaryArtifactsOverride: '', secondarySourcesOverride: '', secondarySourcesVersionOverride: '', serviceRoleOverride: '', sourceControlType: 'jenkins', sourceLocationOverride: 'codebuild-bjs-input-bucket/helloBooot.zip', sourceTypeOverride: 'S3', sourceVersion: '', sseAlgorithm: 'AES256', workspaceExcludes: '', workspaceIncludes: '', workspaceSubdir: ''

codepipeline集成jenkins

codepipeline创建一个四阶段管道

项目结构如下图所示

在这里插入图片描述

以上架构中，由于中国区不支持在pipeline中配置github作为source，所以将codecommit作为source阶段，支持jenkins的只有build和test阶段

在这里插入图片描述

在jenkins安装codepipeline插件

在这里插入图片描述

codepipeline侧配置

按照官方示例创建一个三阶段pipeline（简单起见，跳过测试阶段）

配置build阶段为jenkins，填写参数如下

provider需要和jenkins插件配置一致，url可以填写ec2的ip:port

在这里插入图片描述

jenkins侧配置

主要是安装codepipeline插件，关键配置如下

aksk是为jenkins插件授权查询codepipeline任务状态的，provider需要和codepipeline中配置一致

在这里插入图片描述

配置触发器为每分钟轮询，这样jenkins插件会每分钟轮询codepipeline状态，检测构建任务的触发

在这里插入图片描述

轮询日志如下，如果没有权限会报错

正常结果

Started on Jan 19, 2023, 2:07:00 PM
[AWS CodePipeline Plugin] Polling for jobs for action type id: [Owner: Custom, Category: Build, Provider: testJenkins, Version: 1, ProjectName: mypipeline]
[AWS CodePipeline Plugin] No jobs found.
Done. Took 48 sec
No changes

插件没有凭证的结果

Started on Jan 19, 2023, 2:08:00 PM
[AWS CodePipeline Plugin] Polling for jobs for action type id: [Owner: Custom, Category: Build, Provider: testJenkins, Version: 1, ProjectName: mypipeline]
ERROR: Failed to record SCM polling for hudson.model.FreeStyleProject@341d4f5c[mypipeline]
hudson.model.Failure: 
AWS CodePipeline Jenkins plugin setup error. One or more required configuration parameters have not been specified.
The AWS credentials provided are not valid.

插件配置错误区域的报错

Started on Jan 19, 2023, 2:16:00 PM
[AWS CodePipeline Plugin] Polling for jobs for action type id: [Owner: Custom, Category: Build, Provider: testJenkins, Version: 1, ProjectName: mypipeline]
ERROR: Failed to record SCM polling for hudson.model.FreeStyleProject@341d4f5c[mypipeline]
com.amazonaws.services.codepipeline.model.ActionTypeNotFoundException: ActionType (Category: 'Build', Owner: 'Custom', Provider: 'testJenkins', Version: '1') is not available (Service: AWSCodePipeline; Status Code: 400; Error Code: ActionTypeNotFoundException; Request ID: e4a70f8a-e125-43d0-9467-e6c4b5292840; Proxy: null)

codepipeline的build阶段等待10分钟超时，应该是插件没有识别到pipeline任务

Failed to upload output artifact(s): The number of output artifacts in the Jenkins project and in the pipeline action do not match. Configure the output locations of your Jenkins project to match the pipeline action's output artifacts. Number of output locations in Jenkins project: 0, number of output artifacts in the pipeline action: 1 [Pipeline: test-jenkins, stage: Build, action: Build].

检查配置后重新触发，可见jenkins插件识别到任务，并将s3的输出内容下载到本地的/home/ec2-user/.jenkins/workspace/mypipeline目录，但是仍旧有错误

[AWS CodePipeline Plugin] Job 'c257736e-6cb2-4d00-ba38-fad6b44aba49' received
[AWS CodePipeline Plugin] Acknowledged job with ID: c257736e-6cb2-4d00-ba38-fad6b44aba49
[AWS CodePipeline Plugin] Clearing workspace '/home/ec2-user/.jenkins/workspace/mypipeline' before download
[AWS CodePipeline Plugin] Detected compression type: Zip
[AWS CodePipeline Plugin] Successfully downloaded artifact from AWS CodePipeline
[AWS CodePipeline Plugin] Extracting '/home/ec2-user/.jenkins/workspace/mypipeline/RtMO8D2' to '/home/ec2-user/.jenkins/workspace/mypipeline'
[AWS CodePipeline Plugin] Artifact uncompressed successfully
...

构建完成的日志如下，但是artifact出现错误

[INFO] Building war: /home/ec2-user/.jenkins/workspace/mypipeline/target/unicorn-web-project.war
[INFO] WEB-INF/web.xml already added, skipping
...
[AWS CodePipeline Plugin] Publishing artifacts
[AWS CodePipeline Plugin] The number of output artifacts in the Jenkins project and in the pipeline action do not match. Configure the output locations of your Jenkins project to match the pipeline action's output artifacts. Number of output locations in Jenkins project: 0, number of output artifacts in the pipeline action: 1 [Pipeline: test-jenkins, stage: Build, action: Build].

jenkins插件的逻辑和codepipeline一致，最终会将构建结果回传到s3桶

在这里插入图片描述

测试阶段仍旧报错，需要配置codepipeline的输入为buildartifact，否则该阶段获取的是源码包

[AWS CodePipeline Plugin] The number of output artifacts in the Jenkins project and in the pipeline action do not match. Configure the output locations of your Jenkins project to match the pipeline action's output artifacts. Number of output locations in Jenkins project: 1, number of output artifacts in the pipeline action: 0 [Pipeline: test-jenkins, stage: Test, action: test].
[AWS CodePipeline Plugin] Stacktrace:

在这里插入图片描述