aws codepipeline 在pipeline构建过程中使用变量

参考资料

Action structure reference
codebuild构建环境中的环境变量
codepipeline中的变量
在codePipeline中使用变量

对于codepipeline来说，管道结构中的每个操作都有自身的结构和定义，本文主要讨论不同资源的输出变量。

基本概念

变量允许用户在执行操作时通过自定义的方式配置管道操作。变量可以通过操作执行生成，也可以在每个管道执行开始时隐式提供

变量

例如对于作为source的ecr来说，输出变量包括

RegistryId
RepositoryName
ImageTag
ImageDigest
ImageURI

对于codebuild来说，输出变量需要在buildspec.yaml中单独导出

有的管道结构本身并没有输出变量，例如codedeploy

命名空间

命名空间提供了变量的隔离性，确保唯一引用变量，指定命名空间引用变量

#{namespace.variable_key}

命名空间有两种类型

预留命名空间，例如codeppeline提供的隐式命名空间codepipeline
```
#{codepipeline.PipelineExecutionId}
```
自定义命名空间，对某个操作分配命名空间之后操作生成的变量都在该命名空间下，需要确保命名孔家年在管道定义中唯一
```
#{SourceVariables.VersionId}
```

pipeline actions可用的变量

预定义的变量包括以下结构，无法添加和修改变量输出

CodePipeline execution ID output variable
Amazon ECR action output variables
AWS CloudFormation StackSets action output variables
CodeCommit action output variables
CodeStarSourceConnection action output variables
GitHub action output variables (GitHub action version 1)
S3 action output variables

自定义变量键的结构有以下

CloudFormation action output variables
CodeBuild action output variables
Lambda action output variables

在pipeline中输出和引用变量

新建一个3阶段的pipeline

source阶段为codecommit，source阶段的命名空间为namespace: SourceVariables，
build阶段在管道中配置EnvironmentVariables，可以直接引用上一步管道结构中的环境变量。此外codebuild项目中设置的环境变量和从pipeline中获得的环境变量在具体的构建过程中是一样的。
deploy阶段指定输出到s3桶

$ aws codepipeline get-pipeline --name test-variable
metadata:
  pipelineArn: arn:aws-cn:codepipeline:cn-north-1:xxxxxxxxxx:test-variable
pipeline:
  artifactStore: ...
  name: test-variable
  roleArn: arn:aws-cn:iam::xxxxxxxxxx:role/AWSCodePipelineServiceRole
  stages:
  - actions:
  	name: Source
    - actionTypeId:
        category: Source
        provider: CodeCommit
      name: Source
      namespace: SourceVariables
  - actions:
  	name: Build
    - actionTypeId:
        category: Build
        provider: CodeBuild
      configuration:
        EnvironmentVariables: '[{"name":"pipelineid","value":"#{codepipeline.PipelineExecutionId}","type":"PLAINTEXT"},{"name":"commitid","value":"#{SourceVariables.CommitId}","type":"PLAINTEXT"}]'
        ProjectName: test-variable-build
      inputArtifacts:
      - name: SourceArtifact
      name: Build
      namespace: BuildVariables
      outputArtifacts:
      - name: BuildArtifact
  - actions:
  	name: Deploy
    - actionTypeId:
        category: Deploy
        provider: S3
      configuration:
        BucketName: zhaojiew-temptest
        Extract: 'false'
        ObjectKey: test-variable
      inputArtifacts:
      - name: BuildArtifact
      name: Deploy
      namespace: DeployVariables

source阶段

codecommit阶段输出的变量如下，不需要额外配置

在这里插入图片描述

build阶段

如果在codebuild中配置环境变量，是拿不到codepipeline的命名空间中的变量的

例如，通过printenv查看codebuild的环境变量

在这里插入图片描述

commitid-build=#{SourceVariables.CommitId}
myvariable2=bar
myvariable3=***
pipelineid=5827591f-c4b1-4497-9ec5-a92601078027
...
/codebuild/output/tmp/env.sh: line 101: export: `commitid-build': not a valid identifier

codebuild阶段可以自定义变量导出。codebuild阶段通过exported-variables将变量导出到pipeline中在下一阶段使用。这里直接将自定义的环境变量导出，设定命名空间为BuildVariables

在buildspec.yaml中增加以下配置

version: 0.2
env:
  variables:
    myvariable-shell: "shell-var"
  exported-variables:
    - myvariable2

在这里插入图片描述

综上可知，codebuild可以在三个地方增加环境变量

codebuild项目
codepipeline
buildspec.yaml中（控制台看不到）

简单例子

manule approve中引用管道变量

查看官方的示例，在source阶段之后添加手动审批，获取对应的管道变量，当然可以直接去pipelie的构建历史中不同action中直接查看变量输出

在这里插入图片描述

pipeline会解析变量并生成链接和commit信息

在这里插入图片描述

如果变量不存在汇报以下错误，所以需要注意阶段设置正确

Invalid action configuration
An action in this pipeline failed because one or more variables could not be resolved: Action name=approve. This can result when a variable is referenced that does not exist. Validate the configuration for this action.

codebuild获取ssm parameter

默认情况下，在build阶段打印的环境变量如下

CODEBUILD_SOURCE_VERSION=arn:aws-cn:s3:::codepipeline-cn-north-1-482183469511/docker-to-ecs/SourceArti/Oh7wxuY
HOSTNAME=b13665a8f415
SBT_VERSION=1.6.2
ANDROID_HOME=/usr/local/android-sdk-linux
ANDROID_SDK_EXTRAS=extras;android;m2repository extras;google;m2repository extras;google;google_play_services
POWERSHELL_VERSION=6.2.6
ANT_DOWNLOAD_SHA512=2287dc5cfc21043c14e5413f9afb1c87c9f266ec2a9ba2d3bf2285446f6e4ccb59b558bf2e5c57911a05dfa293c7d5c7ad60ac9f744ba11406f4e6f9a27b2403
RUBY_27_VERSION=2.7.6
AWS_CONTAINER_CREDENTIALS_RELATIVE_URI=/v2/credentials/3fca43bf-4391-433a-a493-3e057990cbb3
GOLANG_13_VERSION=1.13.15
DOCKER_CHANNEL=stable
CODEBUILD_RESOLVED_SOURCE_VERSION=fe103331df4b6aca6b31649da02392049d50794e
N_SRC_DIR=/n
ANDROID_SDK_BUILD_TOOLS_28=build-tools;28.0.3
CODEBUILD_BUILD_SUCCEEDING=1
OLDPWD=/codebuild/readonly
JAVA_11_HOME=/usr/lib/jvm/java-11-amazon-corretto.x86_64
ANDROID_SDK_PLATFORM_TOOLS=platforms;android-29
CODEBUILD_PROJECT_UUID=ec86e5af-e89d-4ae1-8612-012b4e0807e8
GRADLE_VERSION=5.6.4
JRE_HOME=/usr/lib/jvm/java-11-amazon-corretto.x86_64
GITVERSION_VERSION=5.3.5
CODEBUILD_AUTH_TOKEN=06e8863a-51eb-46a3-ada0-10f95d51a01e
CODEBUILD_LOG_PATH=cc9e8db1-b0ed-4266-ae1e-3bd636cbef45
MAVEN_VERSION=3.6.3
INSTALLED_GRADLE_VERSIONS=4.10.3 5.6.4
POWERSHELL_DOWNLOAD_URL=https://github.com/PowerShell/PowerShell/releases/download/v6.2.6/powershell-6.2.6-linux-x64.tar.gz
PHP_73_VERSION=7.3.33
CODEBUILD_BUILD_URL=https://cn-north-1.console.amazonaws.cn/codebuild/home?region=cn-north-1#/builds/test-build-docker-to-ecs:cc9e8db1-b0ed-4266-ae1e-3bd636cbef45/view/new
SBT_DOWNLOAD_SHA256=637637b6c4e6fa04ab62cd364061e32b12480b09001cd23303df62b36fadd440
LOG4J_UNSAFE_VERSIONS=2.11.1 1.2.8
AWS_EXECUTION_ENV=AWS_ECS_EC2
DOCKER_BUCKET=download.docker.com
DIND_COMMIT=3b5fac462d21ca164b3778647420016315289034
MAVEN_OPTS=-Dmaven.wagon.httpconnectionManager.maxPerRoute=2
RUBY_26_VERSION=2.6.10
CODEBUILD_GOPATH=/codebuild/output/src296449416
NUGET_XMLDOC_MODE=skip
PYTHON_39_VERSION=3.9.12
PATH=/usr/local/bin/sbt/bin:/root/.goenv/shims:/root/.goenv/bin:/go/bin:/root/.phpenv/shims:/root/.phpenv/bin:/root/.pyenv/shims:/root/.pyenv/bin:/root/.rbenv/shims:/usr/local/rbenv/bin:/usr/local/rbenv/shims:/root/.dotnet/:/root/.dotnet/tools/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/tools:/usr/local/android-sdk-linux/tools:/usr/local/android-sdk-linux/tools/bin:/usr/local/android-sdk-linux/platform-tools:/codebuild/user/bin
MAVEN_HOME=/opt/maven
AWS_DEFAULT_REGION=cn-north-1
POWERSHELL_DOWNLOAD_SHA=ee5512d869ab9bd59bf17f417ff93013e0a169db91cf848ba2570d4818e05e17
PYYAML_VERSION=5.4.1
DOTNET_ROOT=/root/.dotnet
PWD=/codebuild/output/src296449416/src
CODEBUILD_BUILD_IMAGE=aws/codebuild/amazonlinux2-x86_64-standard:3.0
JAVA_HOME=/usr/lib/jvm/java-11-amazon-corretto.x86_64
GRADLE_DOWNLOADS_SHA256=abc10bcedb58806e8654210f96031db541bcd2d6fc3161e81cb0572d6a15e821 5.6.4\n336b6898b491f6334502d8074a6b8c2d73ed83b92123106bd4bf837f04111043 4.10.3
GOLANG_14_VERSION=1.14.15
CODEBUILD_FE_REPORT_ENDPOINT=https://codebuild.cn-north-1.amazonaws.com.cn/
CODEBUILD_KMS_KEY_ID=arn:aws-cn:kms:cn-north-1:xxxxxxxxxx:alias/aws/s3
GOLANG_12_VERSION=1.12.17
AWS_REGION=cn-north-1
JRE_8_HOME=/usr/lib/jvm/java-1.8.0-amazon-corretto.x86_64/jre
ANDROID_SDK_MANAGER_VER=4333796
PHP_74_VERSION=7.4.29
CODEBUILD_BUILD_ARN=arn:aws-cn:codebuild:cn-north-1:xxxxxxxxxx:build/test-build-docker-to-ecs:cc9e8db1-b0ed-4266-ae1e-3bd636cbef45
PYTHON_PIP_VERSION=21.1.2
CODEBUILD_AGENT_ENDPOINT=http://127.0.0.1:7831
JDK_8_HOME=/usr/lib/jvm/java-1.8.0-amazon-corretto.x86_64
CODEBUILD_CI=true
CODEBUILD_BUILD_ID=test-build-docker-to-ecs:cc9e8db1-b0ed-4266-ae1e-3bd636cbef45
CODEBUILD_CONTAINER_NAME=default
JDK_HOME=/usr/lib/jvm/java-11-amazon-corretto.x86_64
HOME=/root
SHLVL=4
ANDROID_SDK_MANAGER_SHA256=92ffee5a1d98d856634e8b71132e8a95d96c83a63fde1099be3d86df3106def9
CODEBUILD_INITIATOR=codepipeline/docker-to-ecs
NODE_12_VERSION=12.22.12
ANDROID_SDK_BUILD_TOOLS=build-tools;29.0.3
MAVEN_DOWNLOAD_SHA512=c35a1803a6e70a126e80b2b3ae33eed961f83ed74d18fcd16909b2d44d7dada3203f1ffe726c17ef8dcca2dcaa9fca676987befeadc9b9f759967a8cb77181c0
CODEBUILD_SRC_DIR=/codebuild/output/src296449416/src
RUBY_BUILD_SRC_DIR=/usr/local/rbenv/plugins/ruby-build
DOCKER_SHA256=9ccfc39305ae1d8882d18c9c431544fca82913d6df717409ac2244ac58c4f070
CODEBUILD_BUILD_NUMBER=8
GOPATH=/go:/codebuild/output/src296449416
JDK_11_HOME=/usr/lib/jvm/java-11-amazon-corretto.x86_64
GRADLE_PATH=/gradle
DOCKER_VERSION=20.10.15
PYTHON_38_VERSION=3.8.13
DOTNET_31_SDK_VERSION=3.1.419
ECS_CONTAINER_METADATA_URI=http://169.254.170.2/v3/dd120308-6d7b-43cd-a59d-258a5acb4aae
RBENV_SRC_DIR=/usr/local/rbenv
GOENV_DISABLE_GOPATH=1
ANT_VERSION=1.10.12
PYTHON_37_VERSION=3.7.13
CODEBUILD_LAST_EXIT=0
ECS_CONTAINER_METADATA_URI_V4=http://169.254.170.2/v4/dd120308-6d7b-43cd-a59d-258a5acb4aae
CODEBUILD_START_TIME=1672473450442
CODEBUILD_EXECUTION_ROLE_BUILD=
JRE_11_HOME=/usr/lib/jvm/java-11-amazon-corretto.x86_64
DOCKER_COMPOSE_VERSION=1.26.0
CODEBUILD_BMR_URL=https://CODEBUILD_AGENT:3000
JAVA_8_HOME=/usr/lib/jvm/java-1.8.0-amazon-corretto.x86_64
NODE_10_VERSION=10.24.1
ANDROID_SDK_PLATFORM_TOOLS_28=platforms;android-28
_=/usr/bin/printenv

在codebuild中配置获取ssm参数，当然也可以在管道变量中设置

在这里插入图片描述

配置build阶段环境变量来自ssm，需要配置额外权限

Phase context status code: Decrypted Variables Error Message: AccessDeniedException: User: arn:aws-cn:sts::xxxxxxxxxx:assumed-role/codebuild-test-variable-build-service-role/AWSCodeBuild-332a3dac-030c-4bac-9954-1faa40185aa1 is not authorized to perform: ssm:GetParameters on resource: arn:aws-cn:ssm:cn-north-1:xxxxxxxxxx:parameter/hello because no identity-based policy allows the ssm:GetParameters action

codebuild获取source阶段变量

这个之前实际上已经做过了，只需要在build阶段配置环境变量如下

EnvironmentVariables: '[{"name":"pipelineid","value":"#{codepipeline.PipelineExecutionId}","type":"PLAINTEXT"},{"name":"commitid","value":"#{SourceVariables.CommitId}","type":"PLAINTEXT"}]'

最终能够在build详情里看到解析的变量

在这里插入图片描述

获取并使用来自cloudforamtion的变量

Tutorial: Create a pipeline that uses variables from AWS CloudFormation deployment actions

cfn的变量输出是根据堆栈中的outputs部分指定的生辰的。

Note that the only CloudFormation action modes that generate outputs are those that result in creating or updating a stack, such as stack creation, stack updates, and change set execution.

引用语法如下